-
Balancing Kafka on JBOD
At Automattic we run a diverse array of systems and as with many companies Kafka is the glue that ties them together; letting us to shuffle data back and forth. Our experience with Kafka have thus far been fantastic, it’s stable, provides excellent throughput, and the simple API makes it trivial to hook any of our systems up to it.…
-
Log Analysis With Hive
At Automattic we see over 131M unique visitors per month from the US alone. As part of the data team we are responsible for taking in the stream of Nginx logs and turning them into counts of views and unique visitors per day, week, and month on both a per blog and global basis. To…
-
Static Asset Caching Using Apache on Heroku
There’s been many articles written about how to properly implement static asset caching over the years and the best practices boil down three things. Make sure the server is sending RFC compliant caching headers. Send long expires headers for static assets. Use version numbers in asset paths so that we can precisely control expiration.
-
Proxies & IP Spoofing
Back in the good old days of client-server architectures you had one person sitting in front of a browser connecting directly to your server. So if you ever wanted to know who that person is all you needed to do was to take a look at the REMOTE_ADDR of your requests.
-
Building Modules on Heroku
I’ve run this site on Heroku for a while now but something has always bothered me, the default Apache install on the Cedar stack does not come with mod_deflate forcing us to send everything sent to clients uncompressed. Now I get that bandwidth is cheap but CPU time with zlib is cheaper and there’s no reason to…