preg leak

php.internals

Jason Donald

22 years ago
Hi guys, I'm looking for the maintainer of the preg_match function in PHP. There appears to be a nasty leak in its most basic functionality, and being a very fundamental function of PHP (not to mention that my long running scripts use it a lot ;) I thought I'd try and go the direct route. Sorry, I don't know C or I'd try to fix it myself! Considering its nature I thought best to at least make a post to the list rather than it get lost in the bugs pile (the original (much less refined) report from someone else had been sitting a long time). <? while (1) { $body = "any string"; $rand = "any different strings".mt_rand(0,mt_getrandmax()); $pattern = "/$rand/"; preg_match($pattern, $body, $match); } ?> http://bugs.php.net/bug.php?id=28513 This leaks 50MB per second on my PHP5.0.0,5.0.1. It is probably the similar bug reported in PHP4. If you have any suggestions, please let me know! Sincere regards, Jason. _______________________________ Do you Yahoo!? Win 1 of 4,000 free domain names from Yahoo! Enter now. http://promotions.yahoo.com/goldrush

Ilia A.

22 years ago
This is not a bug, but rather expected behavior. PCRE extension caches compiled regular expressions so that subsequent runs of the same regex do not need to perform the compilation step. In your example you are generating new regex in an unterminated loop, so it's no surprise that PHP eventually exhausts the available memory and terminates. Ilia On August 24, 2004 08:28 am, Jason wrote:

John Coggeshall

22 years ago
Shouldn't you be able to disable that cache though? John On Tue, 2004-08-24 at 11:53, Ilia Alshanetsky wrote:

Unnamed Person

22 years ago
Ilia Alshanetsky <ilia@prohost.org> writes:
> This is not a bug, but rather expected behavior. PCRE extension caches > compiled regular expressions so that subsequent runs of the same regex do > not need to perform the compilation step. In your example you are generating > new regex in an unterminated loop, so it's no surprise that PHP eventually > exhausts the available memory and terminates.
Since PHP never knows what the user might do or how long the application might run for, perhaps the cache, a useful feature in this case, should have a maximum cache size. If the maximum cache size is exceeded, the oldest (ideally) cached compiled regexp would be deleted from the cache. It's probably reasonable to keep only a very small number of compiled regular expressions in cache. Intuition, at least, tells me that if a regular expression isn't reused "soon" the compile time is likely not a big deal. I'm guessing that the regular expressions are maintained in such an order that the requested one can be found quickly (via a hash? binary search?). Given my earlier assumption that only a small number really need be cached, they could instead be kept in FIFO order, and a simple linear search of the (small) list done to see if the requested regexp is cached. When it's not found, the one at tail of the queue (assuming the queue is full) would be deleted to make room for a new one which would be pushed onto the head of the queue. Since my assumption is based purely on intuition, is there any indication from "real life" that in fact, keeping many regexps in the cache is truly beneficial? Cheers, Derrell

Ron Korving

22 years ago
FIFO, with the addition that one that's re-used, will be moved to the beginning of the list, would (I think) greatly benefit the cache hit-rate. Just my $0.02 Ron "Derrell Lipman" <Derrell.Lipman@UnwiredUniverse.com> wrote in message news:8yc4a2u3.fsf@random.internal...
> Ilia Alshanetsky <ilia@prohost.org> writes: > > > This is not a bug, but rather expected behavior. PCRE extension caches > > compiled regular expressions so that subsequent runs of the same regex
do
> > not need to perform the compilation step. In your example you are
generating
> > new regex in an unterminated loop, so it's no surprise that PHP
eventually
> > exhausts the available memory and terminates. > > Since PHP never knows what the user might do or how long the application
might
> run for, perhaps the cache, a useful feature in this case, should have a > maximum cache size. If the maximum cache size is exceeded, the oldest > (ideally) cached compiled regexp would be deleted from the cache. > > It's probably reasonable to keep only a very small number of compiled
regular
> expressions in cache. Intuition, at least, tells me that if a regular > expression isn't reused "soon" the compile time is likely not a big deal. > > I'm guessing that the regular expressions are maintained in such an order
that
> the requested one can be found quickly (via a hash? binary search?).
Given
> my earlier assumption that only a small number really need be cached, they > could instead be kept in FIFO order, and a simple linear search of the
(small)
> list done to see if the requested regexp is cached. When it's not found,
the
> one at tail of the queue (assuming the queue is full) would be deleted to
make
> room for a new one which would be pushed onto the head of the queue. > > Since my assumption is based purely on intuition, is there any indication
from

Jason Donald

22 years ago
Thanks for the feedback. A way to somehow limit the caching of expressions would be ideal, otherwise, the memory is lost forever with no way to reclaim it. I imagine a limited size cache which keeps only the most used expressions would be ideal. If anyone can tell me which source file has the 'caching' code in it that would be very appreciated. Jase --- Ron Korving <r.korving@xit.nl> wrote:
> FIFO, with the addition that one that's re-used, > will be moved to the > beginning of the list, would (I think) greatly > benefit the cache hit-rate. > > Just my $0.02 > > Ron > > "Derrell Lipman" > <Derrell.Lipman@UnwiredUniverse.com> wrote in > message > news:8yc4a2u3.fsf@random.internal... > > Ilia Alshanetsky <ilia@prohost.org> writes: > > > > > This is not a bug, but rather expected behavior. > PCRE extension caches > > > compiled regular expressions so that subsequent > runs of the same regex > do > > > not need to perform the compilation step. In > your example you are > generating > > > new regex in an unterminated loop, so it's no > surprise that PHP > eventually > > > exhausts the available memory and terminates. > > > > Since PHP never knows what the user might do or > how long the application > might > > run for, perhaps the cache, a useful feature in > this case, should have a > > maximum cache size. If the maximum cache size is > exceeded, the oldest > > (ideally) cached compiled regexp would be deleted > from the cache. > > > > It's probably reasonable to keep only a very small > number of compiled > regular > > expressions in cache. Intuition, at least, tells > me that if a regular > > expression isn't reused "soon" the compile time is > likely not a big deal. > > > > I'm guessing that the regular expressions are > maintained in such an order > that > > the requested one can be found quickly (via a > hash? binary search?). > Given > > my earlier assumption that only a small number > really need be cached, they > > could instead be kept in FIFO order, and a simple > linear search of the > (small) > > list done to see if the requested regexp is > cached. When it's not found, > the > > one at tail of the queue (assuming the queue is > full) would be deleted to > make > > room for a new one which would be pushed onto the > head of the queue. > > > > Since my assumption is based purely on intuition, > is there any indication > from > > "real life" that in fact, keeping many regexps in > the cache is truly > > beneficial? > > > > Cheers, > > > > Derrell > > -- > PHP Internals - PHP Runtime Development Mailing List > To unsubscribe, visit: http://www.php.net/unsub.php > >
__________________________________ Do you Yahoo!? Yahoo! Mail is new and improved - Check it out! http://promotions.yahoo.com/new_mail

Rasmus Lerdorf

22 years ago
Andrei already fixed it. -Rasmus On Tue, 24 Aug 2004, Jason wrote:

George Schlossnagle

22 years ago
On Aug 24, 2004, at 7:51 PM, Jason wrote:
> Thanks for the feedback. > > A way to somehow limit the caching of expressions > would be ideal, otherwise, the memory is lost forever > with no way to reclaim it. I imagine a limited size > cache which keeps only the most used expressions would > be ideal. > > If anyone can tell me which source file has the > 'caching' code in it that would be very appreciated.
The entirety of the code is in ext/pcre/php_pcre.c, and the cache itself is maintained pretty much entirely in pcre_get_compiled_regex_ex(). George