Wednesday, September 21, 2011

FastCGI with a PHP opcode cache benchmarks

 
 

Sent to you by Danny via Google Reader:

 
 

via Brandon's Blog by Brandon on 7/28/09

In my previous post, I described how to implement FastCGI with a PHP opcode cache on an Apache webserver. My primary motivation for moving to FastCGI was to take advantage of the extra security provided by FastCGI with suEXEC over mod_php. In this post, I'll compare the two environments and provide a few benchmark results.

Benchmark Setup

To compare mod_php to FastCGI with a PHP opcode cache, I used the ab tool on my desktop running PHP 5.3.0, mod_fastcgi 2.4.6 and Apache 2.2.11 on Gentoo linux. Apache had a limited number of modules enabled (actions alias authz_host dav deflate dir expires filter headers log_config mime rewrite setenvif status vhost_alias). The hardware consisted of a Intel Core2 Quad Q6600 (2 x 4MB L2 cache) with 2GB of RAM.

I chose to benchmark against the main page of a stock WordPress 2.8.2 install with no plugins enabled. I chose WordPress because it's extremely easy to set up and many people are familiar with it. I've read that WordPress is very liberal with queries to the database (especially when plugins are involved). Without database queries, I would expect to see a bigger difference between results with an opcode cache and results without one. That being said, most PHP applications today make database queries as part of a typical request and thus I think these results are realistic of what you might experience on your server.

I executed the following command:

$ ab -n 1000 -c 30 http://localhost/

in four different environments:

  • mod_php with a 30MB APC opcode cache (Apache MPM prefork)
  • mod_php without an opcode cache (Apache MPM prefork)
  • mod_fastcgi, suEXEC with a 30MB APC opcode cache (Apache MPM worker)
  • mod_fastcgi, suEXEC without an opcode cache (Apache MPM worker)

Between each benchmark, I reconfigured and restarted the server. I then loaded the main page in a web browser to prime the APC cache.

Benchmark Results

Requests
Graph displaying requests per second of each benchmark scenario

Requests per second

The graph above shows that mod_php and FastCGI behaved very similar in terms of requests per second. Using an opcode cache increased the request rate by almost 3 times in both scenarios. As I mentioned earlier, I'd expect to see an even bigger increase with fewer database queries.

I saw similar results when increasing the number of requests to 10,000, or changing the concurrency level. I never saw any failed requests. I was happy to see there wasn't a significant performance difference moving from mod_php to FastCGI.

Memory Usage
Graph displaying memory usage of each benchmark scenario

Memory usage

The graph above shows the approximate memory usage for each of the 4 benchmarking scenarios. This data was gathered by running the ps_mem.py script immediately after the ab tool completed. This script outputs the private and shared memory usage for each program running on the system. Because I ran my benchmarks in isolation, with nothing else accessing the host, these results should be pretty accurate.

The results surprised me. From the graph you can see that disabling the opcode cache resulted in more memory usage. I think this is misleading. Typically, Linux keeps memory resident until it is needed by another process. I believe the increased memory usage for the non-opcode cache scenarios is actually discarded memory that the Linux kernel hasn't reclaimed. Since each process is having to recompile the PHP script on each request, you see a lot more built up memory reported as private but most likely free to be reused. I suspect running this on a low memory server would yield slightly different results.

You can clearly see the difference in memory usage for the Apache processes (green in the graph) between FastCGI and mod_php. The FastCGI Apache processes are much leaner. Since Apache doesn't need the PHP bloat, it can serve static content much more effectively and quickly.

The scenarios with an opcode cache each used a similar amount of RAM. There is a catch to these results however: I only benchmarked a single user. In a shared environment, each user gets their own PHP instance and opcode cache. With mod_php, all users share the same. Thus I would expect every user to roughly double the RAM used in a FastCGI scenario, while mod_php memory usage would stay relatively constant. This increased memory usage is the main trade-off when going with FastCGI.

Raw Results

For those who want even more detail, below are the raw results from my benchmark tests:

mod_php without an opcode cache:

 This is ApacheBench, Version 2.3 <$Revision: 655654 $> Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/ Licensed to The Apache Software Foundation, http://www.apache.org/  Benchmarking localhost (be patient)  Server Software:        Apache Server Hostname:        localhost Server Port:            80  Document Path:          / Document Length:        5474 bytes  Concurrency Level:      30 Time taken for tests:   28.507 seconds Complete requests:      1000 Failed requests:        0 Write errors:           0 Total transferred:      5717000 bytes HTML transferred:       5474000 bytes Requests per second:    35.08 [#/sec] (mean) Time per request:       855.200 [ms] (mean) Time per request:       28.507 [ms] (mean, across all concurrent requests) Transfer rate:          195.85 [Kbytes/sec] received  Connection Times (ms)               min  mean[+/-sd] median   max Connect:        0    0   0.2      0       2 Processing:   138  849 214.5    842    1608 Waiting:      138  848 214.3    842    1601 Total:        139  849 214.4    843    1608  Percentage of the requests served within a certain time (ms)   50%    843   66%    911   75%    975   80%   1004   90%   1112   95%   1230   98%   1355   99%   1426  100%   1608 (longest request) 

mod_php with opcode cache:

 This is ApacheBench, Version 2.3 <$Revision: 655654 $> Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/ Licensed to The Apache Software Foundation, http://www.apache.org/  Benchmarking localhost (be patient)  Server Software:        Apache Server Hostname:        localhost Server Port:            80  Document Path:          / Document Length:        5474 bytes  Concurrency Level:      30 Time taken for tests:   10.784 seconds Complete requests:      1000 Failed requests:        0 Write errors:           0 Total transferred:      5717000 bytes HTML transferred:       5474000 bytes Requests per second:    92.73 [#/sec] (mean) Time per request:       323.530 [ms] (mean) Time per request:       10.784 [ms] (mean, across all concurrent requests) Transfer rate:          517.70 [Kbytes/sec] received  Connection Times (ms)               min  mean[+/-sd] median   max Connect:        0    0   0.2      0       2 Processing:    66  319  62.9    316     580 Waiting:       66  318  62.9    316     580 Total:         67  319  62.8    316     580  Percentage of the requests served within a certain time (ms)   50%    316   66%    339   75%    354   80%    364   90%    395   95%    424   98%    464   99%    482  100%    580 (longest request) 

FastCGI without an opcode cache:

 This is ApacheBench, Version 2.3 <$Revision: 655654 $> Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/ Licensed to The Apache Software Foundation, http://www.apache.org/  Benchmarking localhost (be patient)  Server Software:        Apache Server Hostname:        localhost Server Port:            80  Document Path:          / Document Length:        5474 bytes  Concurrency Level:      30 Time taken for tests:   28.170 seconds Complete requests:      1000 Failed requests:        0 Write errors:           0 Total transferred:      5695000 bytes HTML transferred:       5474000 bytes Requests per second:    35.50 [#/sec] (mean) Time per request:       845.104 [ms] (mean) Time per request:       28.170 [ms] (mean, across all concurrent requests) Transfer rate:          197.43 [Kbytes/sec] received  Connection Times (ms)               min  mean[+/-sd] median   max Connect:        0    0   0.2      0       2 Processing:   105  832  82.9    832    1004 Waiting:      105  832  82.9    832    1004 Total:        107  832  82.8    832    1004  Percentage of the requests served within a certain time (ms)   50%    832   66%    856   75%    873   80%    882   90%    908   95%    927   98%    954   99%    971  100%   1004 (longest request) 

FastCGI with opcode cache:

 This is ApacheBench, Version 2.3 <$Revision: 655654 $> Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/ Licensed to The Apache Software Foundation, http://www.apache.org/  Benchmarking localhost (be patient)  Server Software:        Apache Server Hostname:        localhost Server Port:            80  Document Path:          / Document Length:        5474 bytes  Concurrency Level:      30 Time taken for tests:   10.917 seconds Complete requests:      1000 Failed requests:        0 Write errors:           0 Total transferred:      5695000 bytes HTML transferred:       5474000 bytes Requests per second:    91.60 [#/sec] (mean) Time per request:       327.509 [ms] (mean) Time per request:       10.917 [ms] (mean, across all concurrent requests) Transfer rate:          509.44 [Kbytes/sec] received  Connection Times (ms)               min  mean[+/-sd] median   max Connect:        0    0   0.2      0       2 Processing:    46  323  36.6    325     426 Waiting:       46  323  36.6    325     426 Total:         47  323  36.5    325     426  Percentage of the requests served within a certain time (ms)   50%    325   66%    335   75%    342   80%    346   90%    359   95%    368   98%    383   99%    389  100%    426 (longest request) 

Memory Usage (in MB):

Benchmark Apache PHP Total
private shared total private shared total
mod_php, no cache 202.3 7.7 210 210
mod_php, with cache 45.3 15.5 60.8 60.8
FastCGI, no cache 9.2 3.3 12.5 68.3 5.0 73.3 85.8
FastCGI, with cache 9.2 3.6 12.8 23.4 13.8 37.2 50

Conclusion

I run my blog on a Linode VPS with a very limited amount of RAM. I also host other applications and a few friends' sites on this same server. Thus, I am very concerned about memory usage. Still, the security advantages provided by FastCGI far outweigh the increased RAM usage.

I hope you have seen that the performance of FastCGI with an opcode cache is just as good as with mod_php. It's also evident how important an opcode cache is to running PHP scripts.

If you have any thoughts about these benchmarks, or the performance of FastCGI vs mod_php, feel free to leave a comment or question below!


 
 

Things you can do from here:

 
 

No comments: