A while back, I discussed the caching used on DailyKos with a hacked up mod_magnet and a lua script to serve up pages for anonymous users out of memcached, so it would avoid hitting the much larger mod_perl apaches on the backend running Scoop. It worked very well, but I never liked having all the extra overhead of mod_magnet when I was only using a small amount of its functionality.
Last week, I finally sat down with Google at my fingertips and a copy of K&R and banged out a standalone module to serve up pages out of memcached. It's been up and running on the site for several days now, and has been performing admirably.
Since it seems to be working pretty well, I've made a patch for it which you can download from here. A README file is included to help you set it up.
Advantages over the hacked up mod_magnet we were using before:
- Doesn't require installing lua
- Doesn't have as much overhead as mod_magnet.
- Uses the actively developed libmemcached instead of the older libmemache.
Disadvantages of mod_mcpage:
- Requires installing libmemcached, which may or may not be available in package form from your distribution.
- Not flexible like mod_magnet - mod_mcpage does exactly one thing, while mod_magnet can do all sorts of neat stuff.
Things to do:
- Currently mod_mcpage only supports "text/html". It should support others.
- There should be an option to support MD5 hashing of URLs for memcached keys, if they're likely to be too long.
- There are a number of memached behaviors that can be set that mod_mcpage doesn't support yet. It should be able to do so.
- Debug statements should be moved out of #ifdefs and be controlled by a configuration variable.
Most of what I've written here won't make sense or be applicable for a lot of you. However, I'm putting it out there in the hopes that it will be useful to some of you. Serving up pages out of memcached for anonymous users has been an absolute godsend for Daily Kos, and if you have a site that's straining under its traffic and are able and comfortable with making changes to your server, then this module may help you a lot.
comments powered by Disqus