PHP Performance tip: require versus require_once

PHP Performance tip: require versus require_once

One of the big performance oriented complaints with PHP is that it doesn’t do well with large frameworks that have a lot of included files. Symfony and Mediawiki are two that I’ve had this problem with.

Why is it slow to load a lot of files in PHP?

Let’s take a closer look.

Quick note: In this post I’ll assume you are already using a PHP Accelerator, such as APC, or Turk MMCache or eaccelerator. If not, you need to be. My personal pick is APC, mostly because it’s the preferred one at Yahoo!, which has the largest installation of PHP, and the author of PHP works there. The lead maintainer of APC also works there, so I feel good knowing that APC is well supported. There are rumors that PHP 6 will have this accelerator built in, and that it will be based on APC’s code.

With that PHP accelerator plug out of the way, let’s get back to business.

Normally when php does a require to include a file, it does a stat to see if the file has changed, and if not, loads it from the APC cache. Here’s what that looks like at the C level:

* stat64("./classes/Class1.php", {st_mode=S_IFREG|0644, st_size=2057, ...}) = 0

Tip: Want to know how to look at what code is doing at the C level? Check out this tutorial on using strace to debug web apps

Nice, simple, clean, one stat per file. Note: with APC, you can set apc.stat to off, and this will skip the above stat call as well. The downside: You have to restart apache whenever you change your code.

Now let’s take a look at what happens when you use require_once instead of require:

* lstat64("/home", {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0
* lstat64("/home/webuser", {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0
* lstat64("/home/webuser/src", {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0
* lstat64("/home/webuser/src/mediawiki", {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0
* lstat64("/home/webuser/src/mediawiki/include_test", {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0
* lstat64("/home/webuser/src/mediawiki/include_test/classes", {st_mode=S_IFDIR|0755, st_size=20480, ...}) = 0
* lstat64("/home/webuser/src/mediawiki/include_test/classes/Class1.php", {st_mode=S_IFREG|0644, st_size=2058, ...}) = 0

That’s one stat for each directory. It does this for every single file you include. With require_once, php must call realpath (at the C level) to know what the actual path of the file is. Otherwise, it won’t know if require_once '../../../mydir/Class.php'; is the same as require_once '../mydir/Class.php';

Note that it also must do this for every directory in your include_path, so if you don’t have that set up correctly, this is exacerbated even more. Each one of these stats is a system call that takes time. More work for your servers and slower responses for your users.

Theory: The extra stats required for require_once and include_once introduce a lot of overhead for applications that include a lot of files.

A real world test — At Wikia, we had a common include file that was loading all of our Mediawiki extensions. It had 113 calls to require_once and 172 calls to include_once. By changing these to require and include respectively, the results were significant.

First, strace revealed that there were 2848 syscalls to serve a page, down from 4782, (-40%). Next I went to ab for more testing, and found that the average page request time went down to 36.5, from 46.5ms (-22%), and the server was able to serve 27.1 requests per second, up from 21.4 (+%22) . View the complete output from ab

Conclusion: require_once does not perform as well as require. Don’t use require_once unless you need it. require will save system calls and deliver pages faster to end users. This also applies to the include/include_once counterparts.

It would be great if someone would write up a tool that walked through your code base and made recommendations for these types of performance tweaks. Hmm….

9 Comments php, software, technology
9 Responses to “PHP Performance tip: require versus require_once”

1.

Clint Byrum on 01 Aug 2008 at 2:46 pm #

You don’t mention the realpath cache and/or its effect (added in php 5.2.0 …). Did you try just setting the realpath cache size bigger?

http://us2.php.net/manual/en/ini.core.php#ini.realpath-cache-size

16K isn’t much room if you have hundreds of files, and lots of directories in your include_path.
2.

nick on 01 Aug 2008 at 4:03 pm #

Good point Clint - I had assumed that everyone would already have realpath_cache_size set higher.

I talked to Rasmus about this recently at OSCON, and he said that realpath_cache definitely helps (epsecially for NFS), but that they can’t cache *misses*, so it still has to stat all of the stats for files and directories that don’t exist — the ones that it had to go through while searching your include_path.

So it’s important to realpath_cache_size set higher for large apps, but the above benchmarks are still valid.
3.

Tony on 07 Aug 2008 at 5:35 am #

Nick:

How does this affect performance if we used absolute paths for require_once()?

How did you solve the conflicts of multiple files ‘require’ the same library?

file1.php - require(’lib1.php’);
file2.php -
require(’lib1.php’);
require(’file1.php’);
4.

nick on 07 Aug 2008 at 6:57 am #

Great question Tony. It seems like more followup testing is in order to come up with a "best practice" way of including files. Other things to consider:

) realpath_cache_size (as Clint pointed out)
) include vs require?
) PHP Autoloading
) Roll your own auto-loading (Although Rasmus warned me that this is the worst, because it uses the less efficient type of caching with APC)
) Behaviour of APC vs other accelerators

Hmm. Maybe I shouldn’t do another write-up…

And as far as resolving conflicts, in the case above I chose that particular file because I knew it was only called once, so I could be 99% sure that it wouldn’t load all those files again. It was a "load all these extensions" type file. It will be much more work to walk through the rest of the application.

One idea I had was to use something like this at the top of the files that were being included:
class_exists(’ClassInFile’) && return;
or this:
function_exists(’functionInFile’) && return;

That way if you accidentally re-included it, the file wouldn’t be reparsed.

Hopefully this highlights the penalty with require_once though, and developers will think twice the next time, and just use require.
5.

Recent Faves Tagged With "syscalls" : MyNetFaves on 22 Oct 2008 at 11:05 pm #

[…] public links >> syscalls PHP Performance tip: require versus require_once First saved by kbhardwaj | 1 days ago Vincenzo Iozzo: First post, me and my project First […]
6.

Generation 5 » An Awesome Autoloader for PHP on 09 Jan 2009 at 2:19 pm #

[…] Wikia Developer Finds require_once() Slower Than require() Another Developer Finds Little Difference Yet Another Developer Finds It Depends On His Cache Configuration Rumor has it, PHP 5.3 improves require_once() performance […]
7.

Konstantin Rozinov on 01 Apr 2009 at 1:11 am #

I ran some tests to see what’s faster: require_once() vs require() and relative_path vs absolute_path.

METHODOLOGY:
The script (test.php):

I ran ab on the test.php script with a different require*() uncommented each time:
ab -n 1000 -c 10 www.example.com/test.php

RESULTS:
(how long it took to execute 1 call to test.php):
require(’absolute_path’): 0.001522074937820
require(’relative_path’): 0.000859447956085
require_once(’absolute_path’): 0.001737765312195
require_once(’relative_path’): 0.000952602624893

When using absolute_path there are fewer stat() system calls.
When using relative_path there are more stat() system calls.

CONCLUSIONS:
Using absolute_path is almost 2 times slower than using relative_path.
Using require_once() is slower than require().

NOTES:
In your blog entry above, when you use relative_path, you only show 1 stat() call. In my case, it stat()ed every directory leading to the relative_path and more…so one of us got wrong results in terms of the number of stat() calls.

Any feedback would be appreciated.

Konstantin Rozinov
8.

Konstantin Rozinov on 01 Apr 2009 at 1:13 am #

The script (test.php) is:

function getmicrotime()
{
list($x,$y) = explode(’ ‘,microtime());
return ($x+$y);
}

$start_time = getmicrotime();

/*
* Uncomment one at a time.
* sql_server.inc only contains define() statements.
*/

//require(’/www/includes/example.com/code/conf/sql_servers.inc’);
//require(’../../includes/example.com/code/conf/sql_servers.inc’);
//require_once(’/www/includes/example.com/code/conf/sql_servers.inc’);
//require_once(’../../includes/example.com/code/conf/sql_servers.inc’);

$end_time = getmicrotime();

$handle = fopen("/tmp/results", "ab+");
fwrite($handle, ($end_time - $start_time) . "\n");
fclose($handle);

Konstantin Rozinov
9.

Konstantin Rozinov on 01 Apr 2009 at 6:13 pm #

Nick,

I have updated results and conclusion to report. Same test.php script and ab command as above, but different, more realistic, results.

RESULTS:
——–
The average time it took to run test.php once:
require(’absolute_path’): 0.000830569960420
require(’relative_path’): 0.000829198306664
require_once(’absolute_path’): 0.000832904849136
require_once(’relative_path’): 0.000824960252097

The average was computed by eliminating the 100 slowest and 100 fastest times, so a total of 800 (1000 - 200) times were used to compute the average time. This was done to eliminate any unusual spikes or dips.

The question of how many stat() system calls were made can be answered as follows:
- If you run httpd -X and then do an strace -p , you can view the system calls that take place to process the request.
- The most important thing to note is if you run test.php continuously (as the ab test does above), the stat() calls only happen for the first request:

first call to test.php (above):
——————————-
lstat64 ("/www", {st_mode=S_IFDIR|0755, st_size=…
lstat64 ("/www/includes", {st_mode=S_IFDIR|0755,…
lstat64 ("/www/includes/example.com", {st_mode=S…
lstat64 ("/www/includes/example.com/code", {st_m…
lstat64 ("/www/includes/example.com/code/conf", …
lstat64 ("/www/includes/example.com/code/conf/sql_servers.inc", {st_mode…
open ("/www/includes/example.com/code/conf/sql_servers.inc", O_RDONLY) = 17

subsequent calls to test.php:
—————————–
open ("/www/includes/example.com/code/conf/sql_servers.inc", O_RDONLY) = 17

- The lack of stat() system calls in the subsequent calls to test.php only happens when test.php is called continusly. If you wait a certain period of time (about 1 minute or so), the stat() calls will happen again.
- This indicates that either the OS (Ubuntu Linux in my case), or Apache is "caching" or knows the results of the previous stat() calls, so it doesn’t bother repeating them.
- When using absolute_path there are fewer stat() system calls.
- When using relative_path there are more stat() system calls because it has to start stat()ing from the current directory back up to / and then to the include/ directory.

CONCLUSIONS:
————
- Try to use absolute_path when calling require*().
- The time difference between require_once() vs. require() is so tiny, it’s almost always insignificant in terms of performance. The one exception is if you have a very large application that has hundreds of require*() calls.
- When using APC opcode caching, the speed difference between the two is completely irrelevant.
- Use an opcode cache, like APC!

Konstantin Rozinov
krozinov [at] gmail