Time To Pull The Plug

This is a subtitle. There are many like it, but this one is here.

Perl Tiered Cache Benchmarks

These benchmarks are for the Perl tiered caching modules discussed here originally. Ruby tiered cache benchmarks shall follow later.

All tests performed on a Xen virtual machine with 512MB of RAM using one core of an Intel Core2 Duo E8500 @ 3.16GHz. The Perl version is 5.10.0 running on Debian 5.0. The remote memcached server is 1.2.5, local memcached is 1.2.2. The modules tested were Cache::Memcached::Fast, Cache::Mmap, Cache::FastMmap, and the custom Cache::Tiered and Cache::TieredFast modules (modules and benchmarking code are here.

The tiered configurations all use a remote memcached server. Cache::Tiered using a local Cache::Mmap cache, Cache::TieredFast uses a local Cache::FastMmap cache, and Cache::TieredLmc uses a local memcached server.

Benchmarking results below the fold. The first set of benchmarks is read-heavy, setting a value and reading it back 50,000 times (and resetting it if need be), the second set of benchmarks set and read a value 50,000 times. The test value used is the zlib HOWTO html file. NB: Right now, with the tiered cache modules, you pass in the memcached or mmap objects you want to use. Eventually, they will make them themselves.

Update:Forgot to add that the mmap files are stored in /dev/shm, but any memory based filesystem should perform the same. Disk-based mmap were not tested, but they don’t seem as likely to give the same speed benefit.

## Read-heavy benchmarks

Using memcached…
Memcache set…
At 10000
At 20000
At 30000
At 40000
At 50000
Memcached elapsed time: 101.018182 seconds : cache misses: 3
Using local memcached…
Local Memcache set…
At 10000
At 20000
At 30000
At 40000
At 50000
local memcached elapsed time: 3.473889 seconds : cache misses: 0
Using mmap…
Mmap set…
At 10000
At 20000
At 30000
At 40000
At 50000
Mmap elapsed time: 5.370702 seconds : cache misses: 0
Using mmap fast…
Mmap fast set…
At 10000
At 20000
At 30000
At 40000
At 50000
Mmap fast elapsed time: 3.16509 seconds : cache misses: 0
Using Cache::Tiered…
Cache::Tiered set…
At 10000
At 20000
At 30000
At 40000
At 50000
Tiered elapsed time: 9.488398 seconds : cache misses: 0
Using Cache::TieredFast…
Cache::TieredFast set…
At 10000
At 20000
At 30000
At 40000
At 50000
TieredFast elapsed time: 5.193063 seconds : cache misses: 0
Using Cache::TieredLmc…
Cache::TieredLmc set…
At 10000
At 20000
At 30000
At 40000
At 50000
TieredLmc elapsed time: 113.350251 seconds : cache misses: 3
Done with read-heavy benchmarks.

## Write-heavy benchmarks

Using memcached…
At 10000
At 20000
At 30000
At 40000
At 50000
Memcached elapsed time: 129.66146 seconds
Using local memcached…
At 10000
At 20000
At 30000
At 40000
At 50000
Local memcached elapsed time: 11.377581 seconds
Using mmap…
At 10000
At 20000
At 30000
At 40000
At 50000
Mmap elapsed time: 12.923498 seconds
Using mmap fast…
At 10000
At 20000
At 30000
At 40000
At 50000
Mmap fast elapsed time: 7.368855 seconds
Using Cache::Tiered…
At 10000
At 20000
At 30000
At 40000
At 50000
Cache::Tiered elapsed time: 44.16001 seconds
Using Cache::TieredFast…
At 10000
At 20000
At 30000
At 40000
At 50000
Cache::TieredFast elapsed time: 40.125187 seconds
Using Cache::TieredLmc…
At 10000
At 20000
At 30000
At 40000
At 50000
Cache::TieredLmc elapsed time: 149.111689 seconds
Done with write-heavy benchmarks.

Done with benchmarking.

Notes:

All of these results seem to be about what you’d expect, except 1) I didn’t think the local memcached would be faster than Cache::Mmap (but it’s still slower than Cache::FastMmap) and 2) in light of #1, it’s surprising that Cache::TieredLmc with a local memcached cache) would perform so badly. Somehow, the value that’s being returned by the local memcached instance is getting blanked before it gets out of the if loop that fetches it, even though there’s nothing that looks like it should be doing it. I’m including the code in the tarball for the local memcached tiered cache, in case some enterprising soul is able to fix it. I’ve looked, but can’t take the time to figure it out. The mmap versions perform as expected, fetching the value and keeping it around long enough to return without hitting the remote memcached. Tiering a remote memached with the mmap improves performance, which makes the Cache::TieredLmc results so surprising, since the local memcached by itself performs so well.