Everything

php

Home » Project » Newsgroups » php.internals » PHP allocating too much memory »

PHP allocating too much memory

php.internals

Peter LeBrun

10 years ago

Hi -

I'm getting the following error in php7.0.5:

Fatal Error: Allowed memory size of 536870912 bytes exhausted (tried to
allocate 140729445144864 bytes)

If it's relevant, we're using dblib+freetds.

I can't reproduce the issue deterministically; the best I can do is to
repeatedly curl an endpoint and something like 30% of the requests will
fail due to the above error.

The weirdest part about this is that PHP is somehow trying to allocate
140TB of memory.

Has anyone seen anything like this?  Any insights would be appreciated.

Thanks,

Peter

Jordi Boggiano

10 years ago

On 15/06/2016 14:01, Peter LeBrun wrote:

> Hi -
>
> I'm getting the following error in php7.0.5:
>
> Fatal Error: Allowed memory size of 536870912 bytes exhausted (tried to
> allocate 140729445144864 bytes)
>
> If it's relevant, we're using dblib+freetds.
>
> I can't reproduce the issue deterministically; the best I can do is to
> repeatedly curl an endpoint and something like 30% of the requests will
> fail due to the above error.
>
> The weirdest part about this is that PHP is somehow trying to allocate
> 140TB of memory.
>
> Has anyone seen anything like this?  Any insights would be appreciated.

I have hit that due to an opcache bug in handling of constant arrays on
classes.

Something like: class Foo { const BLA = []; }

This was fixed in 5.6.15 [1] which came out before PHP7 though, so I'd
imagine that isn't the exact problem you are hitting, but it might be
related.

[1] https://bugs.php.net/bug.php?id=70795

Cheers

--
Jordi Boggiano
@seldaek - http://seld.be

Thomas Hruska

10 years ago

On 6/15/2016 6:07 AM, Jordi Boggiano wrote:

> On 15/06/2016 14:01, Peter LeBrun wrote:
>> Hi -
>>
>> I'm getting the following error in php7.0.5:
>>
>> Fatal Error: Allowed memory size of 536870912 bytes exhausted (tried to
>> allocate 140729445144864 bytes)
>>
>> If it's relevant, we're using dblib+freetds.
>>
>> I can't reproduce the issue deterministically; the best I can do is to
>> repeatedly curl an endpoint and something like 30% of the requests will
>> fail due to the above error.
>>
>> The weirdest part about this is that PHP is somehow trying to allocate
>> 140TB of memory.
>>
>> Has anyone seen anything like this?  Any insights would be appreciated.

I have!

You wouldn't, by any chance, have ODBC in the mix?  The libraries you
mention seem to indicate that you might be using ODBC.  There is a known
issue with mixing 32-bit and 64-bit binaries with PHP and ODBC.  I don't
remember all the details, but mixing 32-bit and 64-bit binaries will see
things like NULL values as not NULL behind the scenes and do the wrong
thing whereby PHP eventually attempts to allocate a ridiculous amount of
RAM (e.g. a -1 value gets translated badly and assumes that it is the
amount of RAM to allocate).  I've seen allocation requests for upwards
of 14 exabytes of RAM before - the amount varies but is always
laughable.  From what I can tell, NULL values in the query results will
always trigger the problem and, even if you can guarantee that there are
no NULL values in ANY of the query results, the issue still arises at a
pretty significant rate due to bit-depth differences.

Unfortunately, due to the way ODBC itself was created and functions,
there is no easy fix within a mixed binary environment.  To *correctly*
fix the problem, you've got to match the bit-depths of ALL of the
binaries involved - that is, either all 32-bit or all 64-bit - which is
not always possible.  The alternative is to use the ugliest hack ever:
Execute a separate PHP process for running each database query that
continually retries when it fails and just live with the high failure
rates.  Guess which solution I used?  Fortunately, in my case, the
number of queries being run was quite limited and on a set schedule.
You might not be so lucky.

--
Thomas Hruska
CubicleSoft President

I've got great, time saving software that you will find useful.

http://cubiclesoft.com/

Johannes Schlueter

10 years ago

Hi,

On Wed, 2016-06-15 at 09:01 -0400, Peter LeBrun wrote:

> I'm getting the following error in php7.0.5:
>
> Fatal Error: Allowed memory size of 536870912 bytes exhausted (tried to
> allocate 140729445144864 bytes)

The huge number 140729445144864 is most likely consequence of an
overflow.

Can you try to reproduce this with a debug build? - Those make the error
message a bit more verbose by stating which emalloc call caused this.
Probably one can receive more info by attaching gdb (or another
debugger) to the PHP process

i.e.

$ gdb -p 1234   # 1234 is the PID of a PHP process
(gdb) source /path/to/php-src/.gdbinit
(gdb) break zend_mm_safe_error
(gdb) run

now hit PHP a few times

(gdb) bt
[dumps system stack trace]
(gdb) zbacktrace
[dumps PHP-level stack trace, gdb macro defined in gdbinit loaded above]


johannes

Rowan Collins

10 years ago

On 15/06/2016 14:01, Peter LeBrun wrote:

> The weirdest part about this is that PHP is somehow trying to allocate
> 140TB of memory.

I've seen numbers like that a few times - always around 140TB, but the
exact number varies. I assume it's an overflow (or underflow?)
somewhere, but the exact mechanism escapes me. (It's close to 2^47, but
not very close; I've got examples logged as "low" as 140090229815192,
and I think I've seen under 140 trillion.)

Apart from sheer curiosity of where this magic number comes from, I
wonder if there is some sanity check missing in the memory manager to at
least display a different error message...

Regards,

--
Rowan Collins
[IMSoP]

Michael Felt

10 years ago

On 15-Jun-16 15:55, Rowan Collins wrote:

> On 15/06/2016 14:01, Peter LeBrun wrote:
>> The weirdest part about this is that PHP is somehow trying to allocate
>> 140TB of memory.
>
> I've seen numbers like that a few times - always around 140TB, but the
> exact number varies. I assume it's an overflow (or underflow?)
> somewhere, but the exact mechanism escapes me. (It's close to 2^47,
> but not very close; I've got examples logged as "low" as
> 140090229815192, and I think I've seen under 140 trillion.)

In hex: 00007F694C627798 - so apparently 00007F69400000000 is common to
all.
FYI: I have seen similar issues with mixed environments (32 and 64-bit)
- at this point I am surprised that you can even dlopen() both sizes (my
OS now refuses to dlopen() 32-bit modules aka shared libraries when
64-bit application and v.v.)

Trevor Suarez

10 years ago

On Wed, Jun 15, 2016 at 11:34 AM, Michael Felt <mamfelt@gmail.com> wrote:

>
>
> On 15-Jun-16 15:55, Rowan Collins wrote:
>
>> On 15/06/2016 14:01, Peter LeBrun wrote:
>>
>>> The weirdest part about this is that PHP is somehow trying to allocate
>>> 140TB of memory.
>>>
>>
>> I've seen numbers like that a few times - always around 140TB, but the
>> exact number varies. I assume it's an overflow (or underflow?) somewhere,
>> but the exact mechanism escapes me. (It's close to 2^47, but not very
>> close; I've got examples logged as "low" as 140090229815192, and I think
>> I've seen under 140 trillion.)
>>
> In hex: 00007F694C627798 - so apparently 00007F69400000000 is common to
> all.
> FYI: I have seen similar issues with mixed environments (32 and 64-bit) -
> at this point I am surprised that you can even dlopen() both sizes (my OS
> now refuses to dlopen() 32-bit modules aka shared libraries when 64-bit
> application and v.v.)
>
>> Apart from sheer curiosity of where this magic number comes from, I
>> wonder if there is some sanity check missing in the memory manager to at
>> least display a different error message...
>>
>> Regards,
>>
>
>
> --
> PHP Internals - PHP Runtime Development Mailing List
> To unsubscribe, visit: http://www.php.net/unsub.php
>
>

I've seen this bug come up with the SOAP extension. I saw it happen when
instantiating a SoapClient or SoapServer when the constructor tries to load
a WSDL file under very certain circumstances. If the SOAP WSDL caching is
on (if `soap.wsdl_cache = 1`), the WSDL file is cached (or is attempting to
be cached) or the WSDL must be downloaded, and the file-system is full,
then this crazy overflow can happen. I believe it's due to the WSDL's
becoming corrupted due to the file-system halting the write of the file and
PHP not cleaning up the improper write.

In fact, this is a reported bug: https://bugs.php.net/bug.php?id=62337

Hope that helps! :)

- Trevor

Peter LeBrun

10 years ago

Hey everyone, thanks for your help and input.  We've narrowed it down to
cases where there is string concatenation with a constant, but currently
upgrading to 7.0.7 to see if that makes a difference.

Enjoy your evening,

Peter

On Wed, Jun 15, 2016 at 1:50 PM, Trevor Suarez <rican7@gmail.com> wrote:

Pierre Joye

10 years ago

Hi

On Jun 16, 2016 7:22 AM, "Peter LeBrun" <peterlebrun@gmail.com> wrote:

>
> Hey everyone, thanks for your help and input.  We've narrowed it down to
> cases where there is string concatenation with a constant, but currently
> upgrading to 7.0.7 to see if that makes a difference.

Is it possible to open a bug or post a reproduce script here please?

> Enjoy your evening,
>
> Peter
>
> On Wed, Jun 15, 2016 at 1:50 PM, Trevor Suarez <rican7@gmail.com> wrote:
>
> > On Wed, Jun 15, 2016 at 11:34 AM, Michael Felt <mamfelt@gmail.com>

wrote:

> >
> > >
> > >
> > > On 15-Jun-16 15:55, Rowan Collins wrote:
> > >
> > >> On 15/06/2016 14:01, Peter LeBrun wrote:
> > >>
> > >>> The weirdest part about this is that PHP is somehow trying to

allocate

> > >>> 140TB of memory.
> > >>>
> > >>
> > >> I've seen numbers like that a few times - always around 140TB, but

the

> > >> exact number varies. I assume it's an overflow (or underflow?)
> > somewhere,
> > >> but the exact mechanism escapes me. (It's close to 2^47, but not very
> > >> close; I've got examples logged as "low" as 140090229815192, and I

think

> > >> I've seen under 140 trillion.)
> > >>
> > > In hex: 00007F694C627798 - so apparently 00007F69400000000 is common

to

> > > all.
> > > FYI: I have seen similar issues with mixed environments (32 and

64-bit) -

> > > at this point I am surprised that you can even dlopen() both sizes

(my OS

> > > now refuses to dlopen() 32-bit modules aka shared libraries when

64-bit

> > > application and v.v.)
> > >
> > >> Apart from sheer curiosity of where this magic number comes from, I
> > >> wonder if there is some sanity check missing in the memory manager

to at

> > >> least display a different error message...
> > >>
> > >> Regards,
> > >>
> > >
> > >
> > > --
> > > PHP Internals - PHP Runtime Development Mailing List
> > > To unsubscribe, visit: http://www.php.net/unsub.php
> > >
> > >
> > I've seen this bug come up with the SOAP extension. I saw it happen when
> > instantiating a SoapClient or SoapServer when the constructor tries to

load

> > a WSDL file under very certain circumstances. If the SOAP WSDL caching

is

> > on (if `soap.wsdl_cache = 1`), the WSDL file is cached (or is

attempting to

> > be cached) or the WSDL must be downloaded, and the file-system is full,
> > then this crazy overflow can happen. I believe it's due to the WSDL's
> > becoming corrupted due to the file-system halting the write of the file

and