Monday, October 19, 2009

Tuning the Apache MaxClients parameter

Published Sun, 2007/03/04 - 18:32, Updated Sat, 2007/10/27 - 13:40

One thing that can have a really drastic effect on a large site using Apache, is the value assigned to the MaxClients parameter.

This parameter defines how many simultaneous request can be served. Any connection request from browsers that come in after that will be queued.

Apache prefork, StartServers, MaxSpareServers and MinSpareServers

In the most common case, you will be using Apache in the prefork mode, meaning one process per connection, with a pool of processes pre-forked to standby for connections. The number of spare processes is defined by the values MaxSpareServers, MinSpareServers, while the number to start is defined by StartServers.

Maxclients default

By default, the MaxClients parameter has a compiled in hard limit of 256. This can be changed by recompiling Apache however. Some distributions, or hosting companies raise this limit to a very high value, such as 512 or even 1024 in order to cope with large loads.

While this makes sense when the web server is serving static content (plain HTML, images, ...etc.), it can be detrimental to a dynamic web application like Drupal. So often, we have clients calling because their web server has grind to a halt, and the reason would be a too high MaxClients value.

A web site's nemesis: Excessive Thrashing

The reason is that if your web site experiences a traffic spike, or if there is a bottleneck in the database, incoming requests cause new processes to be forked at a rate higher than old processes can service the older connections. This causes a condition where the system keeps creating new processes that overflow the available memory and starts to use the swap space. This almost always causes thrashing, where the system is just swapping pages from physical memory to virtual memory (on disk), and vice versa, without doing any real work. You can detect if thrashing has occurred by using the vmstat command (see our page on tools for performance tunings and optimization for more info).

A simple calculation for MaxClients on a system that does only Drupal would be:

(Total Memory - Operating System Memory - MySQL memory) / Size Per Apache process.

If your hosting company configured your server with all sorts of bells and whistles (like mod_perl, mod_python, in addition to mod_php), then Apache can easily be 21 MB per process. If your server has 512MB, then you can fit some 20 Apache processes. If you tune Apache well, and remove all the unneeded modules, and install a PHP op-code cache/accelerator, then you can make each Apache process take as little as 12 MB. These figures depend on how many modules you have loaded, how big they are, so there is no hard and fast rule. Even if one has 1GB of memory, and leaves 250 MB for the system and MySQL, with an Apache process of 15MB, this means 50 Apache processes can fit in the remaining 750MB.

Remember that you need memory for the operating system, as well as for MySQL. The more you give the system and MySQL memory, the more caching of the file system they do for you and avoid hitting disk, so do not use the very last available memory for MaxClients.

Tuning the ServerLimit

On some systems, there is another parameter that  sets an upper limit if MySQL. So for example, if ServerLimit is set by default to 256, and you want to increase MaxClients to 300, you will not be able to do so, until you set ServerLimit to 300 as well. Normally, you would see a warning message from Apache when you restart it to tell you that this needs to be done.

Conclusion

If you cannot do a proper calculation, then it is safest to start with a conservative number, e.g. 60 to 150 on a 2GB system, and then increase it as you monitor the usage of the system over a few weeks. By all means, do not keep it at the 512 value that came with your server/distribution until you know how much load you can handle.

Resources and Links

Apache Performance Tuning. Has a section on MaxClients. Although this document is built with mod_perl in mind, much of it applies to using Apache with PHP as a module.

Performance tuning Apache. Another useful Apache performance tuning document.

How to change the upper limit of MaxClients by recompiling Apache. Not recommended for a dynamic web site.

ProjectOpus blog post by James on Apache2 Maxclients.

Apache performance tuning at devside.net.

‹ Tools for Performance Tuning and Optimization up Using ApacheBench for benchmarking logged in users - an automated approach ›

»

Add new comment

Apache process memory size

Submitted by Visitor (not verified) on Fri, 2009/09/18 - 02:20.

No matter how we try, the process size is always around 60M:

PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND

20808 apache 15 0 362m 58m 21m R 27.4 0.7 0:03.20 httpd

20823 apache 16 0 402m 86m 23m S 10.7 1.1 0:01.67 httpd

20811 apache 16 0 348m 43m 21m S 3.8 0.5 0:02.48 httpd

20826 apache 16 0 362m 52m 21m S 3.4 0.7 0:02.34 httpd

I guess this is also influenced by php memory limit we have? (96M)

»

reply

You probably have too many

Submitted by Khalid on Fri, 2009/09/18 - 09:29.

You probably have too many modules.

Please see: server indigestion: the Drupal contributed modules Open Buffet Binge syndrome.

You can reduce the total memory required on the machine by running PHP as FastCGI, see Apache fcgid: acceptable performance and better resource utilization.

»

reply

Thanks man you really solved

Submitted by ravi (not verified) on Thu, 2009/06/18 - 10:20.

Thanks man you really solved my problem

i am always do wrong

for maxclient

and serverLimit

thanks for share

»

reply

how big are my apache processes?

Submitted by wrb123 (not verified) on Tue, 2009/02/03 - 22:46.

when making this calculation, how do i accurately find my size in MB of each apache process? for example, if i run top -d 1 and look at the results, i have different sizes for each process: VIRT, RES, and SHR. it looks like in top, my total memory usage is in line with SHR, the smallest of the memory numbers. is it safe to make this calculation based on apache process SHR size?

»

reply

what are the required modules for apache?

Submitted by Visitor (not verified) on Tue, 2008/10/21 - 19:28.

Quote:

If your hosting company configured your server with all sorts of bells and whistles (like mod_perl, mod_python, in addition to mod_php), then Apache can easily be 21 MB per process.

== end of quote==

How do I know what apache modules are required to run drupal websites?

we have a dedicated server and root access, and we would like to have the apache modules configured specifically for drupal.

is there a "list" of necessary apache modules that we should have, while disabling all others?

Thanks for such a lovely article

»

reply

Apache modules

Submitted by Khalid on Tue, 2008/10/21 - 19:38.

The only required module is php (mod_php).

In addition to that, having the rewrite module will help your site by using clean URLs.

Also, the deflate module can save you some bandwidth.

--

2bits -- Drupal consulting

»

reply

No comments: