Everything

php

Request-global hashtable initialization

php.internals

Heinz Wiesinger

8 years ago

Hi everyone!

I ventured into writing my first PHP extension and did pretty well so far
following the existing documentation and example code inside the php source.

However, I hit an odd bug in my code that I can't quite explain. I'm obviously
missing something and I hope someone here can point me in the right direction
:)

I'd like to set up a request global hashtable. For now I'm running
ALLOC_HASHTABLE() and zend_hash_init() inside PHP_GINIT_FUNCTION. I then have
a user function that's supposed to add entries to that hashtable and another
user function that reads entries from the hashtable.

I tested with code passed directly on the command line (using php -r) and that
works just like I would expect it too. The value gets written to the hashtable
and I can read it afterwards from the other function. I also verified it's the
same value that I first stored, so from what I can tell the basic
functionality is good.

When I take the same testcode and write it into a file, and then run php on
that file, execution aborts trying to allocate almost 2GB of memory when
trying to store the value in the hashtable (using zend_hash_update()). I can
see PHP_GINIT_FUNCTION being executed in that case as well so I'd assume the
hashtable is setup just fine. Yet something is different.

What's the difference between " php -r '' " and "php test.php" in that regard?

I uploaded my current state of the extension to github in case someone would
like to take a closer look at the code: https://github.com/pprkut/autoload-psr

Thanks in advance!

Grs,
Heinz

Nikita Popov

8 years ago

On Thu, Apr 12, 2018 at 9:53 PM, Heinz Wiesinger <HMWiesinger@liwjatan.at>
wrote:

> Hi everyone!
>
> I ventured into writing my first PHP extension and did pretty well so far
> following the existing documentation and example code inside the php
> source.
>
> However, I hit an odd bug in my code that I can't quite explain. I'm
> obviously
> missing something and I hope someone here can point me in the right
> direction
> :)
>
> I'd like to set up a request global hashtable. For now I'm running
> ALLOC_HASHTABLE() and zend_hash_init() inside PHP_GINIT_FUNCTION. I then
> have
> a user function that's supposed to add entries to that hashtable and
> another
> user function that reads entries from the hashtable.
>
> I tested with code passed directly on the command line (using php -r) and
> that
> works just like I would expect it too. The value gets written to the
> hashtable
> and I can read it afterwards from the other function. I also verified it's
> the
> same value that I first stored, so from what I can tell the basic
> functionality is good.
>
> When I take the same testcode and write it into a file, and then run php on
> that file, execution aborts trying to allocate almost 2GB of memory when
> trying to store the value in the hashtable (using zend_hash_update()). I
> can
> see PHP_GINIT_FUNCTION being executed in that case as well so I'd assume
> the
> hashtable is setup just fine. Yet something is different.
>
> What's the difference between " php -r '' " and "php test.php" in that
> regard?
>
> I uploaded my current state of the extension to github in case someone
> would
> like to take a closer look at the code: https://github.com/pprkut/
> autoload-psr
>
> Thanks in advance!
>
> Grs,
> Heinz
>

https://github.com/pprkut/autoload-psr/blob/27cfc53c864116cf0b65b8f7e69464e15b3fdef5/autoload_psr.c#L175
should be using ZVAL_STR_COPY. The hashtable API does not automatically
incref stored values.

I'd suggest running under USE_ZEND_ALLOC=0 valgrind to check for further
memory issues. Differences in the mode of execution are likely just
incidental.

Nikita

Heinz Wiesinger

8 years ago

On Thursday, 12 April 2018 22:34:52 CEST Nikita Popov wrote:

> On Thu, Apr 12, 2018 at 9:53 PM, Heinz Wiesinger <HMWiesinger@liwjatan.at>
>
> wrote:
> > Hi everyone!
> >
> > I ventured into writing my first PHP extension and did pretty well so far
> > following the existing documentation and example code inside the php
> > source.
> >
> > However, I hit an odd bug in my code that I can't quite explain. I'm
> > obviously
> > missing something and I hope someone here can point me in the right
> > direction
> >
> > :)
> >
> > I'd like to set up a request global hashtable. For now I'm running
> > ALLOC_HASHTABLE() and zend_hash_init() inside PHP_GINIT_FUNCTION. I then
> > have
> > a user function that's supposed to add entries to that hashtable and
> > another
> > user function that reads entries from the hashtable.
> >
> > I tested with code passed directly on the command line (using php -r) and
> > that
> > works just like I would expect it too. The value gets written to the
> > hashtable
> > and I can read it afterwards from the other function. I also verified it's
> > the
> > same value that I first stored, so from what I can tell the basic
> > functionality is good.
> >
> > When I take the same testcode and write it into a file, and then run php
> > on
> > that file, execution aborts trying to allocate almost 2GB of memory when
> > trying to store the value in the hashtable (using zend_hash_update()). I
> > can
> > see PHP_GINIT_FUNCTION being executed in that case as well so I'd assume
> > the
> > hashtable is setup just fine. Yet something is different.
> >
> > What's the difference between " php -r '' " and "php test.php" in that
> > regard?
> >
> > I uploaded my current state of the extension to github in case someone
> > would
> > like to take a closer look at the code: https://github.com/pprkut/
> > autoload-psr
> >
> > Thanks in advance!
> >
> > Grs,
> > Heinz
>
> https://github.com/pprkut/autoload-psr/blob/27cfc53c864116cf0b65b8f7e69464e1
> 5b3fdef5/autoload_psr.c#L175 should be using ZVAL_STR_COPY. The hashtable
> API does not automatically incref stored values.
>
> I'd suggest running under USE_ZEND_ALLOC=0 valgrind to check for further
> memory issues. Differences in the mode of execution are likely just
> incidental.

Thanks for the pointers! I tried running it under valgrind, but couldn't find
anything. So I tried without valgrind, but with USE_ZEND_ALLOC=0 and found
that I can't reproduce the problem like that. Looks to me that something I do
doesn't play well with the zend allocator.

Incidentally valgrind also doesn't tell me about the issue with ZVAL_STR that
you pointed out. Maybe I'm using it wrong? I tried

USE_ZEND_ALLOC=0 valgrind --tool=memcheck --leak-check=full php -
dextension=modules/autoload_psr.so test2.php

Grs,
Heinz

Heinz Wiesinger

8 years ago

On Friday, 13 April 2018 08:16:24 CEST Heinz Wiesinger wrote:

> On Thursday, 12 April 2018 22:34:52 CEST Nikita Popov wrote:
> > On Thu, Apr 12, 2018 at 9:53 PM, Heinz Wiesinger <HMWiesinger@liwjatan.at>
> >
> > wrote:
> > > Hi everyone!
> > >
> > > I ventured into writing my first PHP extension and did pretty well so
> > > far
> > > following the existing documentation and example code inside the php
> > > source.
> > >
> > > However, I hit an odd bug in my code that I can't quite explain. I'm
> > > obviously
> > > missing something and I hope someone here can point me in the right
> > > direction
> > >
> > > :)
> > >
> > > I'd like to set up a request global hashtable. For now I'm running
> > > ALLOC_HASHTABLE() and zend_hash_init() inside PHP_GINIT_FUNCTION. I then
> > > have
> > > a user function that's supposed to add entries to that hashtable and
> > > another
> > > user function that reads entries from the hashtable.
> > >
> > > I tested with code passed directly on the command line (using php -r)
> > > and
> > > that
> > > works just like I would expect it too. The value gets written to the
> > > hashtable
> > > and I can read it afterwards from the other function. I also verified
> > > it's
> > > the
> > > same value that I first stored, so from what I can tell the basic
> > > functionality is good.
> > >
> > > When I take the same testcode and write it into a file, and then run php
> > > on
> > > that file, execution aborts trying to allocate almost 2GB of memory when
> > > trying to store the value in the hashtable (using zend_hash_update()). I
> > > can
> > > see PHP_GINIT_FUNCTION being executed in that case as well so I'd assume
> > > the
> > > hashtable is setup just fine. Yet something is different.
> > >
> > > What's the difference between " php -r '' " and "php test.php" in that
> > > regard?
> > >
> > > I uploaded my current state of the extension to github in case someone
> > > would
> > > like to take a closer look at the code: https://github.com/pprkut/
> > > autoload-psr
> > >
> > > Thanks in advance!
> > >
> > > Grs,
> > > Heinz
> >
> > https://github.com/pprkut/autoload-psr/blob/27cfc53c864116cf0b65b8f7e69464
> > e1 5b3fdef5/autoload_psr.c#L175 should be using ZVAL_STR_COPY. The
> > hashtable API does not automatically incref stored values.
> >
> > I'd suggest running under USE_ZEND_ALLOC=0 valgrind to check for further
> > memory issues. Differences in the mode of execution are likely just
> > incidental.
>
> Thanks for the pointers! I tried running it under valgrind, but couldn't
> find anything. So I tried without valgrind, but with USE_ZEND_ALLOC=0 and
> found that I can't reproduce the problem like that. Looks to me that
> something I do doesn't play well with the zend allocator.
>
> Incidentally valgrind also doesn't tell me about the issue with ZVAL_STR
> that you pointed out. Maybe I'm using it wrong? I tried
>
> USE_ZEND_ALLOC=0 valgrind --tool=memcheck --leak-check=full php -
> dextension=modules/autoload_psr.so test2.php

I moved the hashtable initialization out of PHP_GINIT_FUNCTION and do it on-
demand now in the respective user functions. That seems to have done the
trick, as segfaults and oom messages are gone now.

Grs,
Heinz