Everything

php

[mini-RFC] Disable opcache per script using "declare(cache=0)"

php.internals

Dmitry Stogov

7 years ago

Hi internals,

I like to propose an ability to disable opcache caching per script, using declare(cache=0) at start of PHP file.

The idea was inspired by preloading usage, when some scripts are actually preloading scenarios and have to be executed, but don't have to be cached.

https://github.com/php/php-src/pull/3678

The change is really small in both syntax and implementation, so I wouldn't like to waste time going through the full RFC process.

I'm going to commit this into master on Wednesday (November 28), if no objections.

Thanks. Dmitry.

Nikita Popov

7 years ago

On Fri, Nov 23, 2018 at 11:04 AM Dmitry Stogov <dmitry@zend.com> wrote:

> Hi internals,
>
>
> I like to propose an ability to disable opcache caching per script, using
> declare(cache=0) at start of PHP file.
>
> The idea was inspired by preloading usage, when some scripts are actually
> preloading scenarios and have to be executed, but don't have to be cached.
>
>
> https://github.com/php/php-src/pull/3678
>
>
> The change is really small in both syntax and implementation, so I
> wouldn't like to waste time going through the full RFC process.
>
> I'm going to commit this into master on Wednesday (November 28), if no
> objections.
>
>
> Thanks. Dmitry.
>

Can you please provide an example where this would be used? Should all
scripts involved in preloading (including the main one) be marked this way?
I'm not sure I fully understand the purpose.

Nikita

Dmitry Stogov

7 years ago

Hi Nikita,

Especially, the main preload.php, usually should be marked, to disable its caching.

Thanks. Dmitry.

________________________________
From: Nikita Popov <nikita.ppv@gmail.com>
Sent: Friday, November 23, 2018 1:08:53 PM
To: Dmitry Stogov
Cc: PHP internals
Subject: Re: [PHP-DEV] [mini-RFC] Disable opcache per script using "declare(cache=0)"

On Fri, Nov 23, 2018 at 11:04 AM Dmitry Stogov <dmitry@zend.com<mailto:dmitry@zend.com>> wrote:
Hi internals,

I like to propose an ability to disable opcache caching per script, using declare(cache=0) at start of PHP file.

The idea was inspired by preloading usage, when some scripts are actually preloading scenarios and have to be executed, but don't have to be cached.

https://github.com/php/php-src/pull/3678

The change is really small in both syntax and implementation, so I wouldn't like to waste time going through the full RFC process.

I'm going to commit this into master on Wednesday (November 28), if no objections.

Thanks. Dmitry.

Can you please provide an example where this would be used? Should all scripts involved in preloading (including the main one) be marked this way? I'm not sure I fully understand the purpose.

Nikita

Pierre Joye

7 years ago

Hi Dmitry,

On Fri, Nov 23, 2018, 7:48 PM Dmitry Stogov <dmitry@zend.com wrote:

> Hi Nikita,
>
>
> Especially, the main preload.php, usually should be marked, to disable its
> caching.
>

should this specific case not be done automatically then?

best,

Sara Golemon

7 years ago

On Fri, Nov 23, 2018 at 8:36 AM Pierre Joye <pierre.php@gmail.com> wrote:

> > Especially, the main preload.php, usually should be marked, to disable
> its
> > caching.
> >
>
> should this specific case not be done automatically then?
>
> I'm with Pierre on this.  During the preload phase, we should probably

have implicit caching disabled and only cache things brought in via
opcache_compile_file().
Outside of the preload pragma, I'd worry that this declare would be
mistakenly overused in most cases.  Question though... who wins if a file
is explicitly loaded with opcache_compile_file *and* it has the "don't
cache me, bro" pragma?  I'd expect the file wouldn't be cached in that case.

That said, I'm of the "here's your gun, try not to shoot yourself" school
on most things, and I see no harm in this when used judiciously, so no
objections from me.

-Sara

Rowan Collins

7 years ago

On 23 November 2018 12:48:40 GMT+00:00, Dmitry Stogov <dmitry@zend.com> wrote:

>Especially, the main preload.php, usually should be marked, to disable
>its caching.

Sorry, could you explain for those of us in the peanut gallery why this is the case? I'm struggling to understand what it means to preload something but not cache it, and suspect I'm missing something fundamental.

Regards,

--
Rowan Collins
[IMSoP]

Zeev Suraski

7 years ago

On Sat, Nov 24, 2018 at 1:18 AM Rowan Collins <rowan.collins@gmail.com>
wrote:

> On 23 November 2018 12:48:40 GMT+00:00, Dmitry Stogov <dmitry@zend.com>
> wrote:
> >Especially, the main preload.php, usually should be marked, to disable
> >its caching.
>
> Sorry, could you explain for those of us in the peanut gallery why this is
> the case? I'm struggling to understand what it means to preload something
> but not cache it, and suspect I'm missing something fundamental.
>
>

I'm not sure if you're missing anything fundamental - it's just that the
preload.php file Dmitry's referring to (the one that's responsible to
loading all the other files) - is one file that's pretty much by
definition, will be of no use at any later point in the lifetime of the
server.  It preloads other files (which of course, should very much persist
in the opcache memory) - but once it's done, it's done.  That's why it kind
of makes sense not to cache it.

That said, I'm not sure whether having this feature as a declare() makes
sense - at least in the context of the new preload feature.  The preload
file is likely to be fairly small, wouldn't it consume negligible amounts
of shared memory and just one slot in the files table?  And like Pierre
suggested, perhaps we can simply default to not caching the main preload
file (i.e. the one referenced by php.ini).

Outside the context of preloading, there's probably some value in having
this new declare() option.  There are situations where you simply don't
want to cache certain files (what I usually bumped into was in case of
auto-generated, short-TTL files).  That's why a lifetime or two ago we
added the blacklist feature - the declare() syntax would offer a new way to
do that that is configurationless and built into the code.  In that sense,
it's not a bad thing.  I'm just seeing it as not so much related to
preloading, and more of a general-purpose feature.

Unless of course I'm missing something..?

Zeev

Rowan Collins

7 years ago

On 23/11/2018 23:49, Zeev Suraski wrote:

> I'm not sure if you're missing anything fundamental - it's just that
> the preload.php file Dmitry's referring to (the one that's responsible
> to loading all the other files) - is one file that's pretty much by
> definition, will be of no use at any later point in the lifetime of
> the server.  It preloads other files (which of course, should very
> much persist in the opcache memory) - but once it's done, it's done. 
> That's why it kind of makes sense not to cache it.

Ah, OK, so it's more a case of "don't bother caching" rather than "must
avoid caching" then?

I think making it automatically uncached might be a bit confusing, and a
source of future "gotchas". While it's *logical* for the INI setting to
reference a stub file with nothing but includes, the feature as
described in the RFC doesn't actually *require* this; it would be
perfectly possible for someone to put a bunch of definitions in
"preloaded_functions.php" and reference that one file directly in the
preload directive. On the flip side, it's likely that frameworks will
start shipping with their own preload helpers, so that you'd have
"preload_all.php" reference "symfony/preload.php", and want to exclude
*both* from the cache.

Thinking about it, I'm not clear exactly what a "do not cache" directive
would mean in a preloaded file, since preloading is more than just
caching. For instance, if "class A" is declared in a preloaded file with
declare(cache=0), then "class B extends A" is declared in a preloaded
file with caching enabled, how will class B be treated? Will the file
containing class B be cached, but need to be referenced in an "include"
statement in each request? Again, people will *probably* be using file
structures and autoloaders that make this trivial, but the language
doesn't require that.

Regards,

--
Rowan Collins
[IMSoP]

Stas Malyshev

7 years ago

Hi!

>> I'm not sure if you're missing anything fundamental - it's just that
>> the preload.php file Dmitry's referring to (the one that's responsible
>> to loading all the other files) - is one file that's pretty much by
>> definition, will be of no use at any later point in the lifetime of
>> the server.  It preloads other files (which of course, should very
>> much persist in the opcache memory) - but once it's done, it's done. 
>> That's why it kind of makes sense not to cache it.

Ah, ok, this makes sense. Though usually such file would be rather small
and adding whole new directive just for it may not be that necessary, I
think.

> Thinking about it, I'm not clear exactly what a "do not cache" directive
> would mean in a preloaded file, since preloading is more than just
> caching. For instance, if "class A" is declared in a preloaded file with
> declare(cache=0), then "class B extends A" is declared in a preloaded
> file with caching enabled, how will class B be treated? Will the file

Yes, that's what I am wondering too - how things defined directly in
preload file then will be treated? While file itself is not useful later
in server's lifetime, the things defined in it - like classes or
functions - can be?

--
Stas Malyshev
smalyshev@gmail.com

Marco Pivetta

7 years ago

Adding to the pile of "it's an edge case", since the preload scripts will
be procedural, wouldn't it be sufficient to call
`opcache_invalidate(__FILE__)` at the end of them?


On Sat, 24 Nov 2018, 20:57 Stanislav Malyshev <smalyshev@gmail.com wrote:

Sara Golemon

7 years ago

On Sat, Nov 24, 2018 at 11:03 PM Marco Pivetta <ocramius@gmail.com> wrote:

> Adding to the pile of "it's an edge case", since the preload scripts will
> be procedural, wouldn't it be sufficient to call
> `opcache_invalidate(__FILE__)` at the end of them?

>

I suppose the only negative there would be some very minor fragmentation,
though again, since we can reasonably expect preload scripts to be fairly
simple affairs, I'm not massively bothered by this approach. On the other
hand, I love its simplicity.

Zeev Suraski

7 years ago

On Sun, Nov 25, 2018 at 6:04 PM Sara Golemon <pollita@php.net> wrote:

> On Sat, Nov 24, 2018 at 11:03 PM Marco Pivetta <ocramius@gmail.com> wrote:
>
> > Adding to the pile of "it's an edge case", since the preload scripts will
> > be procedural, wouldn't it be sufficient to call
> > `opcache_invalidate(__FILE__)` at the end of them?
>
>
> >
> I suppose the only negative there would be some very minor fragmentation,
>

One thing that's worth mentioning is that there's no fragmentation ever
with OPcache - because we actually never reclaim freed space (only during a
restart, where, well, we start over completely with a clean slate...).
Once a file is in the cache, the space it consumes is never freed and can
never be (re)used by another file.  Excellent strategy to eliminate
fragmentation, not quite as good for reducing memory consumption though :)

Zeev

Larry Garfield

7 years ago

On Saturday, November 24, 2018 11:03:01 PM CST Marco Pivetta wrote:

> Adding to the pile of "it's an edge case", since the preload scripts will
> be procedural, wouldn't it be sufficient to call
> `opcache_invalidate(__FILE__)` at the end of them?

Are we certain that it will always be procedural?  Is there a part of the
preload script design that prevents it from defining a function?

I'd be very surprised if someone defined a class in a preload script, but I
can definitely see a function getting defined somewhere.  And if it includes
some other file (eg, Composer may be generating one, frameworks may build one,
etc.) then who knows what that file will have defined.

Would that cause issues in this case?

--Larry Garfield

Marco Pivetta

7 years ago

Even if there were definitions in said script, the same issue would arise
with a directive.

On Sun, 25 Nov 2018, 18:35 Larry Garfield <larry@garfieldtech.com wrote:

Zeev Suraski

7 years ago

On Sun, Nov 25, 2018 at 7:03 AM Marco Pivetta <ocramius@gmail.com> wrote:

> Adding to the pile of "it's an edge case", since the preload scripts will
> be procedural, wouldn't it be sufficient to call
> `opcache_invalidate(__FILE__)` at the end of them?
>
> That would actually not do anything useful - as the file will (in all

likelihood) never be accessed again anyway.  Invalidating a file only
ensures it won't be fetched again from the opcache - it doesn't actually
free up the space it consumes.  Invalidate it or not, in both cases the
outcome is the same - a bit of memory consumed by a file that will never
again be fetched from the opcache.  I believe this is what Dmitry's
proposal was about - not putting the file into the opcache to begin with,
to save the bit of memory that it would otherwise consume (but I could be
wrong, as the savings are likely to be so small I'm not sure why we would
care about them...).

Zeev

Marco Pivetta

7 years ago

Is that space rrrrrrreeeeeally a problem?

Take the example ZF loader from the RFC: that barely makes any difference
at all.

A stronger reasoning for another language construct (that changes engine
behaviour) is kinfa required.

On 25 Nov 2018 22:34, "Zeev Suraski" <vsuraski@gmail.com> wrote:

On Sun, Nov 25, 2018 at 7:03 AM Marco Pivetta <ocramius@gmail.com> wrote:

> Adding to the pile of "it's an edge case", since the preload scripts will
> be procedural, wouldn't it be sufficient to call
> `opcache_invalidate(__FILE__)` at the end of them?
>
> That would actually not do anything useful - as the file will (in all

likelihood) never be accessed again anyway.  Invalidating a file only
ensures it won't be fetched again from the opcache - it doesn't actually
free up the space it consumes.  Invalidate it or not, in both cases the
outcome is the same - a bit of memory consumed by a file that will never
again be fetched from the opcache.  I believe this is what Dmitry's
proposal was about - not putting the file into the opcache to begin with,
to save the bit of memory that it would otherwise consume (but I could be
wrong, as the savings are likely to be so small I'm not sure why we would
care about them...).

Zeev

Stas Malyshev

7 years ago

Hi!

> Especially, the main preload.php, usually should be marked, to disable its caching.

Why should it be marked as non-cacheable? I am feeling there's something
important I am missing here. Could you explain a bit more about this?

--
Stas Malyshev
smalyshev@gmail.com