On Tue, Sep 27, 2016 at 9:08 AM, Thomas Lamy <thomas.lamy@netwake.de> wrote:
> Hi,
>
> it's the second time I'm running into this: In a rather big project I'm
> using gearman to enable semi-parallel processing of many small jobs. I'm
> using PHP 5.6.22-0+deb8u1 with OPcache 7.0.6-dev on Debian.
>
> All was well before the code needed refactoring for a new feature: 30
> workers processed 4000 jobs/second at around 5% CPU usage.
> After refactoring (no functional changes), some times after starting the
> workers, processing is getting slower and slower. After about 2 hours I hit
> 100% CPU usage, at about 300 jobs/second.
>
> Memory usage only doubles from start to stall, which is what I observed
> before when everything was fine.
>
>
> I suspect this to be a problem of the cycle collector, or with garbage
> collecting in general. So at any given time, I want PHP to dump all the
> zval's around, which I have to evaluate manually to find "problematic"
> ones. This could be triggered by either a function call, or by
> adding/extending PHP's signal handler and sending that signal to the
> interpreter process.
>
> Where do I have to look at in the sources? Or is there already a tool
> which does what I want, but didn't find yet?
>
>
> I also learned this _could_ be hash collisions related to reordering and
> renaming the data in my code, but as I'm currently stuck with 5.6 I can't
> use the HashDoS related patches floating around here.
>
>
> Any hints on where to find the culprit?
>
>
> Thanks,
>
> Thomas
Before trying anything more sophisticated, I'd suggest to use "perf record"
to check whether the GC is indeed the culprit. If it is, you'll see
functions like "gc_mark_grey" etc taking the majority of the time.
Nikita
