Everything

php

VM reentry limit

php.internals

Nikita Popov

6 years ago

Hi internals,

We have a long-standing issue (tracked at
https://bugs.php.net/bug.php?id=64196 and the very numerous duplicates)
that certain types of infinite recursion can lead to a stack overflow.
While for us it is easy to diagnose this, end users will only see a
"segmentation fault" and will be unable to correlate this with infinite
recursion as the root cause.

To provide some technical context, recursion in PHP usually occurs on the
virtual machine stack, in which case unbounded recursion is supported, as
long as the stack size does not exceed your memory limit. However, some
types of calls (in particular magic methods and certain callbacks) go
through an internal function and have to reenter the virtual machine. This
uses up space on the C stack and may ultimately result in a stack overflow.

I would like to propose the introduction of a zend.vm_reentry_limit ini
option as a solution to this problem, implemented in
https://github.com/php/php-src/pull/5135. This ini setting will limit the
number of nested VM reentries that are allowed before an Error is thrown.

It should be noted that this is (intentionally) not a general recursion
limit. Deep recursion can happen legitimately (e.g. during AST processing)
and it is hard to put a reasonable upper limit on it that both detects
unintentional infinite recursion while allowing legitimate deep recursion.
The limit implemented here exists specifically to prevent stack overflows
and give the programmer a more obvious indication of the cause of the
problem.

Regards,
Nikita

PS: Some extensions will force all function calls to occur via VM reentry.
Such extensions should probably either disable the option (by setting it to
-1), or significantly increase the limit, to avoid false positives.

Ruud Boon

6 years ago

> On 3 Feb 2020, at 14:29, Nikita Popov <nikita.ppv@gmail.com> wrote:
>
> Hi internals,
>
> We have a long-standing issue (tracked at
> https://bugs.php.net/bug.php?id=64196 and the very numerous duplicates)
> that certain types of infinite recursion can lead to a stack overflow.
> While for us it is easy to diagnose this, end users will only see a
> "segmentation fault" and will be unable to correlate this with infinite
> recursion as the root cause.
>
> To provide some technical context, recursion in PHP usually occurs on the
> virtual machine stack, in which case unbounded recursion is supported, as
> long as the stack size does not exceed your memory limit. However, some
> types of calls (in particular magic methods and certain callbacks) go
> through an internal function and have to reenter the virtual machine. This
> uses up space on the C stack and may ultimately result in a stack overflow.
>
> I would like to propose the introduction of a zend.vm_reentry_limit ini
> option as a solution to this problem, implemented in
> https://github.com/php/php-src/pull/5135. This ini setting will limit the
> number of nested VM reentries that are allowed before an Error is thrown.
>
> It should be noted that this is (intentionally) not a general recursion
> limit. Deep recursion can happen legitimately (e.g. during AST processing)
> and it is hard to put a reasonable upper limit on it that both detects
> unintentional infinite recursion while allowing legitimate deep recursion.
> The limit implemented here exists specifically to prevent stack overflows
> and give the programmer a more obvious indication of the cause of the
> problem.
>
> Regards,
> Nikita
>
> PS: Some extensions will force all function calls to occur via VM reentry.
> Such extensions should probably either disable the option (by setting it to
> -1), or significantly increase the limit, to avoid false positives.

I would love to see this addition, I think it can be really helpful for a lot of users.
Can we add a function to report the number of VM reentries as well (get_vm_reentry_count). This could be helpful for determining a reasonable ini value.

Cheers,
Ruud

Nikita Popov

5 years ago

On Mon, Feb 3, 2020 at 2:29 PM Nikita Popov <nikita.ppv@gmail.com> wrote:

> Hi internals,
>
> We have a long-standing issue (tracked at
> https://bugs.php.net/bug.php?id=64196 and the very numerous duplicates)
> that certain types of infinite recursion can lead to a stack overflow.
> While for us it is easy to diagnose this, end users will only see a
> "segmentation fault" and will be unable to correlate this with infinite
> recursion as the root cause.
>
> To provide some technical context, recursion in PHP usually occurs on the
> virtual machine stack, in which case unbounded recursion is supported, as
> long as the stack size does not exceed your memory limit. However, some
> types of calls (in particular magic methods and certain callbacks) go
> through an internal function and have to reenter the virtual machine. This
> uses up space on the C stack and may ultimately result in a stack overflow.
>
> I would like to propose the introduction of a zend.vm_reentry_limit ini
> option as a solution to this problem, implemented in
> https://github.com/php/php-src/pull/5135. This ini setting will limit the
> number of nested VM reentries that are allowed before an Error is thrown.
>
> It should be noted that this is (intentionally) not a general recursion
> limit. Deep recursion can happen legitimately (e.g. during AST processing)
> and it is hard to put a reasonable upper limit on it that both detects
> unintentional infinite recursion while allowing legitimate deep recursion.
> The limit implemented here exists specifically to prevent stack overflows
> and give the programmer a more obvious indication of the cause of the
> problem.
>

Bump.

One open question here is how the error should be reported, when the VM
reentry limit is hit. What my current implementation does is to throw an
Error exception, which is nice in that it contains a stack trace. As this
error condition is caused by infinite recursion, having a stack trace
available is fairly valuable.

On the other hand, using an Error exception can also cause issues, because
they can be caught, and they go through normal unwinding. For example, a
destructor run during unwinding could cause the limit to be hit again,
which would probably lead to some non-obvious behavior.

Regards,
Nikita