No patch yet, but I added a local memcached instance that only stores stuff for a few seconds to mod_mcpage to see how it worked, and got some very interesting benchmarks. My original plan had been to craft up a cache with POSIX shared memory (which is pretty sweet), semaphores, and Glib hashes, but I thought about it some and decided to just let memcached handle the housekeeping stuff, since with a custom shared mem cache I'd have to worry about expiring older stuff and removing unused entries myself.
In summary, though, being served from a paravirtualized one proc Xen VM with 128MB RAM, a local memcached instance of 16MB, and a remote memcached server with 64MB running on a fully virtualized one proc Xen VM with 128MB RAM (processor-wise, the Xen host has a Intel Core2 Duo E8500 @ 3.16GHz):
Running 1000 requests with a concurrency of 1 had 95% of all requests served in 3ms with both local and remote memcacheds, 95% within 17ms with just the remote memcached, and 95% within 780ms with no memcached at all.
Running 1000 requests with a concurrency of 5, with both 95% were within 135ms (80% within 9ms, 50% within 7ms), with just the remote 95% were within 90ms (80% were within 69ms and 50% were within 63ms), and no memcaching at all 95% were within 3766ms (80% were within 3621ms and 50% were within 3554ms).
Mean times for concurrency of one with both, remote only, and no memcached: 4.004ms, 13.878ms, and 778.267ms.
Mean times (across all requests) for concurrency of five with both, remote only, and no memcached: 3.926ms, 13.359ms, and 718.912ms.
Update: As I was cleaning up mod_mcpage to make a patch out of it in the relatively near future, I took out a bunch of random debugging stuff that was going into the log and re-ran ab again, this time with the index page of the site I was testing with using the tiered local + remote caches against the same file being served statically. Over 10,000 requests at a concurrency of one, the tiered cache had a mean time per request of 2.216ms. The static file had a mean time of 2.110ms. Over 10,000 requests with a concurrency of five, the tiered cache had a mean time per request of 1.120ms across all requests. The static file had a mean time of 1.025ms across all requests.
Not too bad. Doing it in shared memory would be blazingly fast, but who knows when I'll have time to get all the little bits of that done. Something worth shooting for down the road though.
Full benchmarks (including the update) below the fold.
Running ab -n 1000 -c 1 xxxx:8888
With both local memcached and remote memcached:
Server Software: lighttpd/1.5.0
Server Hostname: xxxx
Server Port: 8888
Document Path: /
Document Length: 105458 bytes
Concurrency Level: 1
Time taken for tests: 4.004 seconds
Complete requests: 1000
Failed requests: 0
Write errors: 0
Total transferred: 105719000 bytes
HTML transferred: 105458000 bytes
Requests per second: 249.76 [#/sec] (mean)
Time per request: 4.004 [ms] (mean)
Time per request: 4.004 [ms] (mean, across all concurrent requests)
Transfer rate: 25785.48 [Kbytes/sec] received
Connection Times (ms)
min mean[+/-sd] median max
Connect: 0 0 0.6 0 9
Processing: 2 4 13.0 2 169
Waiting: 1 2 13.0 1 167
Total: 2 4 13.0 3 169
Percentage of the requests served within a certain time (ms)
50% 3
66% 3
75% 3
80% 3
90% 3
95% 3
98% 4
99% 75
100% 169 (longest request)
With just the remote memcached:
Server Software: lighttpd/1.5.0
Server Hostname: xxxx
Server Port: 8888
Document Path: /
Document Length: 105458 bytes
Concurrency Level: 1
Time taken for tests: 13.878 seconds
Complete requests: 1000
Failed requests: 0
Write errors: 0
Total transferred: 105719000 bytes
HTML transferred: 105458000 bytes
Requests per second: 72.05 [#/sec] (mean)
Time per request: 13.878 [ms] (mean)
Time per request: 13.878 [ms] (mean, across all concurrent requests)
Transfer rate: 7439.02 [Kbytes/sec] received
Connection Times (ms)
min mean[+/-sd] median max
Connect: 0 0 0.8 0 10
Processing: 11 14 10.0 13 300
Waiting: 9 12 10.0 11 298
Total: 11 14 10.1 13 300
Percentage of the requests served within a certain time (ms)
50% 13
66% 14
75% 14
80% 15
90% 16
95% 17
98% 18
99% 21
100% 300 (longest request)
With no memcaching (ab reports Apache rather than lighttpd because lighttpd is just proxying content back):
Server Software: Apache/2.2.9
Server Hostname: xxxx
Server Port: 8888
Document Path: /
Document Length: 105427 bytes
Concurrency Level: 1
Time taken for tests: 778.267 seconds
Complete requests: 1000
Failed requests: 0
Write errors: 0
Total transferred: 105739000 bytes
HTML transferred: 105427000 bytes
Requests per second: 1.28 [#/sec] (mean)
Time per request: 778.267 [ms] (mean)
Time per request: 778.267 [ms] (mean, across all concurrent requests)
Transfer rate: 132.68 [Kbytes/sec] received
Connection Times (ms)
min mean[+/-sd] median max
Connect: 0 1 2.5 0 20
Processing: 667 778 774.7 709 10898
Waiting: 656 765 774.8 697 10887
Total: 668 778 774.7 710 10899
Percentage of the requests served within a certain time (ms)
50% 710
66% 718
75% 723
80% 729
90% 745
95% 780
98% 911
99% 993
100% 10899 (longest request)
Running ab -n 1000 -c 5 xxxx:8888
With both local memcached and remote memcached:
Server Software: lighttpd/1.5.0
Server Hostname: xxxx
Server Port: 8888
Document Path: /
Document Length: 105458 bytes
Concurrency Level: 5
Time taken for tests: 3.926 seconds
Complete requests: 1000
Failed requests: 0
Write errors: 0
Total transferred: 105719000 bytes
HTML transferred: 105458000 bytes
Requests per second: 254.72 [#/sec] (mean)
Time per request: 19.629 [ms] (mean)
Time per request: 3.926 [ms] (mean, across all concurrent requests)
Transfer rate: 26297.67 [Kbytes/sec] received
Connection Times (ms)
min mean[+/-sd] median max
Connect: 0 1 0.6 1 4
Processing: 3 19 38.6 7 217
Waiting: 1 9 30.9 2 211
Total: 4 20 38.6 7 218
Percentage of the requests served within a certain time (ms)
50% 7
66% 8
75% 9
80% 9
90% 30
95% 135
98% 152
99% 157
100% 218 (longest request)
With just the remote memcached:
Server Software: lighttpd/1.5.0
Server Hostname: xxxx
Server Port: 8888
Document Path: /
Document Length: 105458 bytes
Concurrency Level: 5
Time taken for tests: 13.359 seconds
Complete requests: 1000
Failed requests: 0
Write errors: 0
Total transferred: 105779816 bytes
HTML transferred: 105518555 bytes
Requests per second: 74.86 [#/sec] (mean)
Time per request: 66.794 [ms] (mean)
Time per request: 13.359 [ms] (mean, across all concurrent requests)
Transfer rate: 7732.82 [Kbytes/sec] received
Connection Times (ms)
min mean[+/-sd] median max
Connect: 0 1 0.7 1 10
Processing: 34 66 27.2 63 385
Waiting: 9 35 20.7 35 341
Total: 35 67 27.2 63 385
Percentage of the requests served within a certain time (ms)
50% 63
66% 66
75% 68
80% 69
90% 74
95% 90
98% 110
99% 253
100% 385 (longest request)
With no memcaching (ab reports Apache rather than lighttpd because lighttpd is just proxying content back):
Server Software: Apache/2.2.9
Server Hostname: xxxx
Server Port: 8888
Document Path: /
Document Length: 105427 bytes
Concurrency Level: 5
Time taken for tests: 718.912 seconds
Complete requests: 1000
Failed requests: 0
Write errors: 0
Total transferred: 105739000 bytes
HTML transferred: 105427000 bytes
Requests per second: 1.39 [#/sec] (mean)
Time per request: 3594.560 [ms] (mean)
Time per request: 718.912 [ms] (mean, across all concurrent requests)
Transfer rate: 143.63 [Kbytes/sec] received
Connection Times (ms)
min mean[+/-sd] median max
Connect: 0 0 1.2 0 20
Processing: 678 3577 233.1 3554 7580
Waiting: 668 3565 233.0 3541 7568
Total: 679 3577 233.1 3554 7580
Percentage of the requests served within a certain time (ms)
50% 3554
66% 3582
75% 3608
80% 3621
90% 3681
95% 3766
98% 3868
99% 3999
100% 7580 (longest request)
Running ab -n 10000 -c 1 xxxx:8888/
With the local and remote memcaches, after removing the debugging statements going to the error log:
Server Software: lighttpd/1.5.0
Server Hostname: xxxx
Server Port: 8888
Document Path: /
Document Length: 105458 bytes
Concurrency Level: 1
Time taken for tests: 22.164 seconds
Complete requests: 10000
Failed requests: 0
Write errors: 0
Total transferred: 1056910000 bytes
HTML transferred: 1054580000 bytes
Requests per second: 451.17 [#/sec] (mean)
Time per request: 2.216 [ms] (mean)
Time per request: 2.216 [ms] (mean, across all concurrent requests)
Transfer rate: 46567.37 [Kbytes/sec] received
Connection Times (ms)
min mean[+/-sd] median max
Connect: 0 0 0.5 0 20
Processing: 2 2 0.6 2 26
Waiting: 0 0 0.3 0 11
Total: 2 2 0.8 2 27
Percentage of the requests served within a certain time (ms)
50% 2
66% 2
75% 2
80% 2
90% 2
95% 2
98% 3
99% 4
100% 27 (longest request)
Running ab -n 10000 -c 1 xxxx:8888/testdex.html
The same file as above, but served statically.
Server Software: lighttpd/1.5.0
Server Hostname: xxxx
Server Port: 8888
Document Path: /testdex.html
Document Length: 105428 bytes
Concurrency Level: 1
Time taken for tests: 21.104 seconds
Complete requests: 10000
Failed requests: 0
Write errors: 0
Total transferred: 1056640000 bytes
HTML transferred: 1054280000 bytes
Requests per second: 473.84 [#/sec] (mean)
Time per request: 2.110 [ms] (mean)
Time per request: 2.110 [ms] (mean, across all concurrent requests)
Transfer rate: 48894.47 [Kbytes/sec] received
Connection Times (ms)
min mean[+/-sd] median max
Connect: 0 0 0.4 0 10
Processing: 2 2 0.2 2 12
Waiting: 0 0 0.1 0 10
Total: 2 2 0.4 2 12
Percentage of the requests served within a certain time (ms)
50% 2
66% 2
75% 2
80% 2
90% 2
95% 2
98% 3
99% 3
100% 12 (longest request)
Running ab -n 10000 -c 5 xxxx:8888/
With the local and remote memcaches, after removing the debugging statements going to the error log:
Server Software: lighttpd/1.5.0
Server Hostname: xxxx
Server Port: 8888
Document Path: /
Document Length: 105458 bytes
Concurrency Level: 5
Time taken for tests: 11.200 seconds
Complete requests: 10000
Failed requests: 0
Write errors: 0
Total transferred: 1056910000 bytes
HTML transferred: 1054580000 bytes
Requests per second: 892.85 [#/sec] (mean)
Time per request: 5.600 [ms] (mean)
Time per request: 1.120 [ms] (mean, across all concurrent requests)
Transfer rate: 92154.27 [Kbytes/sec] received
Connection Times (ms)
min mean[+/-sd] median max
Connect: 0 1 1.5 1 30
Processing: 2 5 4.0 4 72
Waiting: 0 1 1.6 1 30
Total: 3 6 5.2 5 91
Percentage of the requests served within a certain time (ms)
50% 5
66% 5
75% 6
80% 6
90% 7
95% 7
98% 8
99% 14
100% 91 (longest request)
Running ab -n 10000 -c 5 xxxx:8888/testdex.html
The same file as above, but served statically.
Server Software: lighttpd/1.5.0
Server Hostname: 10.250.55.15
Server Port: 8888
Document Path: /testdex.html
Document Length: 105428 bytes
Concurrency Level: 1
Time taken for tests: 21.104 seconds
Complete requests: 10000
Failed requests: 0
Write errors: 0
Total transferred: 1056640000 bytes
HTML transferred: 1054280000 bytes
Requests per second: 473.84 [#/sec] (mean)
Time per request: 2.110 [ms] (mean)
Time per request: 2.110 [ms] (mean, across all concurrent requests)
Transfer rate: 48894.47 [Kbytes/sec] received
Connection Times (ms)
min mean[+/-sd] median max
Connect: 0 0 0.4 0 10
Processing: 2 2 0.2 2 12
Waiting: 0 0 0.1 0 10
Total: 2 2 0.4 2 12
Percentage of the requests served within a certain time (ms)
50% 2
66% 2
75% 2
80% 2
90% 2
95% 2
98% 3
99% 3
100% 12 (longest request)
Caveats: ab is not the perfect benchmarking software, these were just meant to get a rough idea how tiering the cache would perform, and it's liable to have more tweaks done before a patch is made.
comments powered by Disqus