[PATCH] suppressing realpath() with APC + tiny patches

php.internals

Masaki Fujimoto

22 years ago
Hello, These days I've been working on some tasks to reduce I/O system calls (lstat64, stat64, open and so on) of PHP for some reasons and as the result of it, I noticed that with the APC + very small two pathes we can suppress most of realpath() calls (although currently I'm not sure this is the correct method or not...): http://eth.jp/php-cvs-20040506-delayed-fopen.patch.txt (patch against current 4.3 tree, because APC is not yet fully compatible with PHP 5) http://eth.jp/apc-2.0.3-delayed-fopen.patch.txt (patch against APC-2.0.3, current CVS-HEAD is somehow unstable...) So please try and review if you're curious about suppressing realpath() calls:) Usage: 1) apply patches to current 4.3 tree, and APC-2.0.3 2) buidconf and configure PHP as always with --enable-delayed-fopen option 3) build and install PHP and APC as always 4) add "delayed_fopen=1" entry to your php.ini And details are as follows: As the recent thread and some other threads show (which listed below), PHP requires quite a few I/O system calls especially when large number of scripts are include_once()ed. And these are not ignorable for the websites with more than 1 million hits per hour although it has nothing to do with small sites. [Performance Consideration of 1 class per file] http://marc.theaimsgroup.com/?l=php-dev&m=108279256414391&w=2 [Performance degradation] http://www.phpbuilder.com/lists/php-developer-list/2003022/0226.php The main concern here is the lstat64() by realpath(), and open() by searching for include_paths (when many include_paths are set). I think suppressing open() (which will result in ENOENT) is not so easy (because we need some other new caching system for include_paths), but perhaps we can easily reduce realpath() for included scritpts with APC, because APC does not use an opned file handle when cache hits. So, my patch skips zend_open() until open_file_scanning() is called inside zend_compile_file() when include_once() or require_once() is called, so that zend_open() is not called when the target script is cached in APC. As a result of this, we can reduce most of realpath() calls by include_once() and required_once(). These patches are not yet verified well so comments are welcome! Best regards, Masaki Fujimoto fujimoto@php.net

Rasmus Lerdorf

22 years ago
> (patch against APC-2.0.3, current CVS-HEAD is somehow unstable...)
How so? -Rasmus

Masaki Fujimoto

22 years ago
The Message From Rasmus Lerdorf on Thu, 6 May 2004 07:58:38 -0700 (Pacific Standard Time):
> > (patch against APC-2.0.3, current CVS-HEAD is somehow unstable...) > > How so?
I haven't yet inpected the cause but APC in CVS crashed with PHP 4.3.7-dev (tested today), so I chose stable APC-2.0.3... Anyway, the patch for APC containts only few lines and is of course available for current CVS-HEAD:)
-- Masaki Fujimoto fujimoto@php.net

Andi Gutmans

22 years ago
Hi Masaki, I would like to fix this using a realpath() cache. I plan on working on this in the next few weeks and this will hopefully solve the problem everywhere in PHP and be pretty much transparent. Andi At 10:36 PM 5/6/2004 +0900, Masaki Fujimoto wrote:

Masaki Fujimoto

22 years ago
Hi, I'm glad to hear that:) Actually, my first try was writing an experimental patch to cache realpath() for this problem because perhaps this is the finest and most "transparent" solution as you say. And my experimental patch worked fine , too (though it's too ugly to post here:). Anyway, the reason I chose the "patch with APC" is just because it's the most simple one (so that it could be acceptable for more people...). So, I'm curious about your design and implementation of realpath() cache, like this: - will it be "inter process" or "inter request" caching system? - if so, php will depend on shared memory or something like that? - or, you will take more simple way (like using /tmp/realpath_cache directory to store the cache) ? - and how to determine when to expire each cache? - if not, it won't be enough for me... (although it's true that it will be ok for most of php users) I'd like to hear your idea:) Masaki Fujimoto The Message From Andi Gutmans:

Andi Gutmans

22 years ago
Hi, I will probably make it per-process, obviously it'll stay alive in between requests with a TTL. I don't think it should take more than a few KB so I wouldn't worry about memory. I might in any case, limit the maximum of memory it can take. Give me a couple of weeks and hopefully I'll have something ready. I think this would be a great new feature! Andi At 06:14 PM 5/10/2004 +0900, Masaki fujimoto wrote:

Magnus Maatta

22 years ago
On Monday 10 May 2004 11.20, Andi Gutmans wrote:
> Hi, > > I will probably make it per-process, obviously it'll stay alive in between > requests with a TTL. I don't think it should take more than a few KB so I > wouldn't worry about memory. I might in any case, limit the maximum of > memory it can take. Give me a couple of weeks and hopefully I'll have > something ready. I think this would be a great new feature!
And make a configure switch to turn the cache off, not all OSes need it. =) /Magnus

Derick Rethans

22 years ago
On Mon, 10 May 2004, Magnus [iso-8859-1] Määttä wrote:
> On Monday 10 May 2004 11.20, Andi Gutmans wrote: > > Hi, > > > > I will probably make it per-process, obviously it'll stay alive in between > > requests with a TTL. I don't think it should take more than a few KB so I > > wouldn't worry about memory. I might in any case, limit the maximum of > > memory it can take. Give me a couple of weeks and hopefully I'll have > > something ready. I think this would be a great new feature! > > And make a configure switch to turn the cache off, not all OSes need it. =)
What would be the point of turning it off? regards, Derick