Everything
php
Learn Community Releases The PHP Project
Home » Project » Newsgroups » php.internals » PCRE caching »

PCRE caching

php.internals

Rasmus Schultz

9 years ago
Hey internals, I was wondering whether or how PCRE regular expression get parsed and cached, and I found this answer on Stack Overflow: http://stackoverflow.com/questions/209906/compile-regex-in-php Do I understand this correctly, that: 1. All regular expressions are hashed and the compiled expression is cached internally between calls. 2. The /S modifier applies more optimizations during compile, but caching works the same way. 3. Compiled expressions are not cached between requests. If so, this seems far from optimal. Every unique regular expression needs to be compiled during every request, right? So with FPM, or with long-running apps, we're missing an opportunity to optimize by caching between requests. And with long-running apps, we're caching every dynamic regular expression, which could harm (memory overhead) more than help. Ideally, shouldn't we have (like some engines/languages) a switch to enable caching? The run-time can't know if a given regular expression is dynamic or static, can it? It's just a string either way - so without a switch, you're either committing compiled dynamic expressions to the cache unnecessarily, and/or missing an opportunity to cache between requests in long-running apps or under FPM. I think most apps use quite a lot of regular expression for validation etc. so maybe there's a missed optimization opportunity here? Cheers, Rasmus Schultz

Ivan Enderlin @ Hoa

9 years ago
Hello, We can also use a LRU caching strategy with a pre-defined (or user-defined) number of expressions to keep in the cache. This would also be a good idea to track number of times a regular expression is used. If this number reaches a certain threshold, then we could automatically re-compile it with the `S` modifier. I agree there is a missed optimisation here. I share your feeling. Regards. On 01.03.17 16:35, Rasmus Schultz wrote:

Nikita Popov

9 years ago
On Wed, Mar 1, 2017 at 4:35 PM, Rasmus Schultz <rasmus@mindplay.dk> wrote:
> Hey internals, > > I was wondering whether or how PCRE regular expression get parsed and > cached, and I found this answer on Stack Overflow: > > http://stackoverflow.com/questions/209906/compile-regex-in-php > > Do I understand this correctly, that: > > 1. All regular expressions are hashed and the compiled expression is cached > internally between calls. >
Correct. 2. The /S modifier applies more optimizations during compile, but caching
> works the same way. >
Yes. Additionally, if PCRE JIT is enabled (which it usually is on PHP 7) we always study, independently of whether /S was specified.
> 3. Compiled expressions are not cached between requests. >
Compiled expressions are cached between requests. However, they are not shared between processes (I'm not even sure if that's possible.) The cache invalidation strategy is FIFO. More specifically, whenever the cache fills up, we discard the first 1/8 cached regular expressions. Nikita

Rasmus Schultz

9 years ago
Thanks for clearing this up, Nikita :-)
> Compiled expressions are cached between requests. However, they are not
shared between processes That sounds good - it would likely very difficult (if not impossible) and likely would make only a marginal performance difference. All good, I think.
> The cache invalidation strategy is FIFO
Okay, so, do cache entries move "to the front of the line" when there's a cache hit? I'm thinking, otherwise, a large number of dynamic expressions (which don't, or only minimally, benefit from caching) might actually push static expressions (which do benefit) out of the cache. Just wondering. If invalidation of static expressions does happen in that case, it likely doesn't impact most apps - I don't think dynamic expressions are very common in most apps, though I have used it occasionally for things like dictionary search... On Fri, Mar 3, 2017 at 10:28 PM, Nikita Popov <nikita.ppv@gmail.com> wrote: