Everything
php
Learn Community Releases The PHP Project
Home » Project » Newsgroups » php.internals » the experimental jit-branch is impressive »

the experimental jit-branch is impressive

php.internals

lists@rhsoft.net

9 years ago
https://github.com/zendtech/php-src/tree/jit-dynasm/ext/opcache/jit that below is a "ab -c 50 -n 100000" on a Intel(R) Core(TM) i7-3770 CPU @ 3.40GHz on our core cms (pgo-build with heavy compiler optimizations, only the source tarball deifferent) currently our production 2x6 core machine with two Intel(R) Xeon(R) CPU E5-2643 v3 @ 3.40GHz and the same setup is around 4000-5000 per second which is nearly reached by the 6 years old i7 with the JIT impressive! JIT: Requests per second: 3526.56 [#/sec] (mean) Time per request: 14.178 [ms] (mean) Time per request: 0.284 [ms] (mean, across all concurrent requests) 7.1: Requests per second: 1419.87 [#/sec] (mean) Time per request: 35.214 [ms] (mean) Time per request: 0.704 [ms] (mean, across all concurrent requests)

lists@rhsoft.net

9 years ago
and with a demo-page containing all sort of modules and bloat the difference is even greater - can't wait to see that in production is there anything known when it is expected to arrive in the official tree or does Zend even hold it back until the point when they can suprise with a "we are done, all tests are fine and we can merge it" announce? PHP 7.1: Requests per second: 316.77 [#/sec] (mean) Time per request: 157.844 [ms] (mean) Time per request: 3.157 [ms] (mean, across all concurrent requests) Transfer rate: 12604.11 [Kbytes/sec] received PHP 7.2 JIT: Requests per second: 925.96 [#/sec] (mean) Time per request: 53.998 [ms] (mean) Time per request: 1.080 [ms] (mean, across all concurrent requests) Transfer rate: 36842.68 [Kbytes/sec] received Am 02.05.2017 um 18:00 schrieb lists@rhsoft.net:

Nikita Popov

9 years ago
On Tue, May 2, 2017 at 7:14 PM, lists@rhsoft.net <lists@rhsoft.net> wrote:
> and with a demo-page containing all sort of modules and bloat the > difference is even greater - can't wait to see that in production > > is there anything known when it is expected to arrive in the official tree > or does Zend even hold it back until the point when they can suprise with a > "we are done, all tests are fine and we can merge it" announce? > > PHP 7.1: > Requests per second: 316.77 [#/sec] (mean) > Time per request: 157.844 [ms] (mean) > Time per request: 3.157 [ms] (mean, across all concurrent requests) > Transfer rate: 12604.11 [Kbytes/sec] received > > PHP 7.2 JIT: > Requests per second: 925.96 [#/sec] (mean) > Time per request: 53.998 [ms] (mean) > Time per request: 1.080 [ms] (mean, across all concurrent requests) > Transfer rate: 36842.68 [Kbytes/sec] received
These results are very unlikely. I'm 95% sure your benchmark is broken. My first guess would be that you're benchmarking PHP + JIT against PHP without opcache. Please share the relevant (opcache-related) portion of the php.inis you used. Nikita

lists@rhsoft.net

9 years ago
Am 02.05.2017 um 20:02 schrieb Nikita Popov:
> On Tue, May 2, 2017 at 7:14 PM, lists@rhsoft.net > <mailto:lists@rhsoft.net> <lists@rhsoft.net <mailto:lists@rhsoft.net>> > wrote: > > and with a demo-page containing all sort of modules and bloat the > difference is even greater - can't wait to see that in production > > is there anything known when it is expected to arrive in the > official tree or does Zend even hold it back until the point when > they can suprise with a "we are done, all tests are fine and we can > merge it" announce? > > PHP 7.1: > Requests per second: 316.77 [#/sec] (mean) > Time per request: 157.844 [ms] (mean) > Time per request: 3.157 [ms] (mean, across all concurrent requests) > Transfer rate: 12604.11 [Kbytes/sec] received > > PHP 7.2 JIT: > Requests per second: 925.96 [#/sec] (mean) > Time per request: 53.998 [ms] (mean) > Time per request: 1.080 [ms] (mean, across all concurrent requests) > Transfer rate: 36842.68 [Kbytes/sec] received > > These results are very unlikely. I'm 95% sure your benchmark is broken. > My first guess would be that you're benchmarking PHP + JIT against PHP > without opcache. Please share the relevant (opcache-related) portion of > the php.inis you used
no they are not - since i build RPM packages and even the whole spec-file is unchanged, only the tarball changed and the build is highly optimized i can assure you for 100% that i compare PHP 7.1.5RC1 with https://github.com/zendtech/php-src downloaded today maybe the PGO-profiling running autotests and fuzzy-calls on the whole application as well as 2000 cms-requests combined with the compiler flags improves the JIT itself without opcache the results for 7.1.5 are *dramatically* slower and i repeated the test upgrade/downgrade packages and run "ab" multiple times on that machine - attached the "php.spec" which is used for the build in just downloaded the zip from https://github.com/zendtech/php-src, renamed it to "php-7.2.0", made a tar.xz archive, changed the version on teh frist line in the spec file and fired the build/profiling - nothing else changed _______________________________________________ phpinfo() of the test machine PHP Version 7.1.5RC1 Build Date May 2 2017 12:19:59 This program makes use of the Zend Scripting Language Engine: Zend Engine v3.1.0, Copyright (c) 1998-2017 Zend Technologies with Zend OPcache v7.1.5RC1, Copyright (c) 1999-2017, by Zend Technologies with Xdebug v2.5.3, Copyright (c) 2002-2017, by Derick Rethans Directive Local Value Master Value opcache.blacklist_filename no value no value opcache.consistency_checks 0 0 opcache.dups_fix Off Off opcache.enable On On opcache.enable_cli Off Off opcache.enable_file_override On On opcache.error_log /var/log/php_error.log /var/log/php_error.log opcache.fast_shutdown 1 1 opcache.file_update_protection 2 2 opcache.force_restart_timeout 180 180 opcache.huge_code_pages On On opcache.inherited_hack On On opcache.interned_strings_buffer 8 8 opcache.lockfile_path /tmp /tmp opcache.log_verbosity_level 1 1 opcache.max_accelerated_files 1000 1000 opcache.max_file_size 327680 327680 opcache.max_wasted_percentage 5 5 opcache.memory_consumption 128 128 opcache.opt_debug_level 0 0 opcache.optimization_level 0x7FFFBFFF 0x7FFFBFFF opcache.preferred_memory_model no value no value opcache.protect_memory 0 0 opcache.restrict_api /usr/share/php/zendoptimizer.php /usr/share/php/zendoptimizer.php opcache.revalidate_freq 5 5 opcache.revalidate_path Off Off opcache.save_comments 0 0 opcache.use_cwd On On opcache.validate_permission Off Off opcache.validate_root Off Off opcache.validate_timestamps On _______________________________________________

lists@rhsoft.net

9 years ago
that would be the numbers of 7.1.5RC1 *without* opcache Requests per second: 136.46 [#/sec] (mean) Time per request: 366.405 [ms] (mean) Time per request: 7.328 [ms] (mean, across all concurrent requests) Transfer rate: 5429.78 [Kbytes/sec] received 7.1.5: Requests per second: 136.46 7.1.5 opcache: Requests per second: 316.77 7.2.0 JIT: Requests per second: 925.96 same hardware, same scripts, httpd hard restarted ab -c 50 -n 20000 believe it or not - i know my php environemt and what i benchmark :-) Am 02.05.2017 um 20:12 schrieb lists@rhsoft.net:

lists@rhsoft.net

9 years ago
and BTW: for the 7.2.0 build i have not built pecl-extensions so the application is runnign witout APCu which means loading a single template in a core-cms-setup takes 1.6% versus 0.06% of the whole page maybe one missing relevant information: * no 3rd party code and micro-optimized to death over months * 100% declare(strict_types=1); after some months of works * 99.9% of all functions are using type-hints for all params could be that the JIT has some benefit from that combined with the PGP-profiling and heavily CPU optimizations - i will give it a try and leave out the PGO-profiling and compare if the gain stays similar Am 02.05.2017 um 20:21 schrieb lists@rhsoft.net:

Nikita Popov

9 years ago
On Tue, May 2, 2017 at 8:12 PM, lists@rhsoft.net <lists@rhsoft.net> wrote:
> > > Am 02.05.2017 um 20:02 schrieb Nikita Popov: > >> On Tue, May 2, 2017 at 7:14 PM, lists@rhsoft.net <mailto:lists@rhsoft.net> >> <lists@rhsoft.net <mailto:lists@rhsoft.net>> wrote: >> >> and with a demo-page containing all sort of modules and bloat the >> difference is even greater - can't wait to see that in production >> >> is there anything known when it is expected to arrive in the >> official tree or does Zend even hold it back until the point when >> they can suprise with a "we are done, all tests are fine and we can >> merge it" announce? >> >> PHP 7.1: >> Requests per second: 316.77 [#/sec] (mean) >> Time per request: 157.844 [ms] (mean) >> Time per request: 3.157 [ms] (mean, across all concurrent requests) >> Transfer rate: 12604.11 [Kbytes/sec] received >> >> PHP 7.2 JIT: >> Requests per second: 925.96 [#/sec] (mean) >> Time per request: 53.998 [ms] (mean) >> Time per request: 1.080 [ms] (mean, across all concurrent requests) >> Transfer rate: 36842.68 [Kbytes/sec] received >> >> These results are very unlikely. I'm 95% sure your benchmark is broken. >> My first guess would be that you're benchmarking PHP + JIT against PHP >> without opcache. Please share the relevant (opcache-related) portion of the >> php.inis you used >> > > no they are not - since i build RPM packages and even the whole spec-file > is unchanged, only the tarball changed and the build is highly optimized i > can assure you for 100% that i compare PHP 7.1.5RC1 with > https://github.com/zendtech/php-src downloaded today > > maybe the PGO-profiling running autotests and fuzzy-calls on the whole > application as well as 2000 cms-requests combined with the compiler flags > improves the JIT itself > > without opcache the results for 7.1.5 are *dramatically* slower > > and i repeated the test upgrade/downgrade packages and run "ab" multiple > times on that machine - attached the "php.spec" which is used for the build > > in just downloaded the zip from https://github.com/zendtech/php-src, > renamed it to "php-7.2.0", made a tar.xz archive, changed the version on > teh frist line in the spec file and fired the build/profiling - nothing > else changed > _______________________________________________ > > phpinfo() of the test machine > > PHP Version 7.1.5RC1 > Build Date May 2 2017 12:19:59 > > This program makes use of the Zend Scripting Language Engine: Zend Engine > v3.1.0, Copyright (c) 1998-2017 Zend Technologies with Zend OPcache > v7.1.5RC1, Copyright (c) 1999-2017, by Zend Technologies with Xdebug > v2.5.3, Copyright (c) 2002-2017, by Derick Rethans > > Directive Local Value Master Value > opcache.blacklist_filename no value no value > opcache.consistency_checks 0 0 > opcache.dups_fix Off Off > opcache.enable On On > opcache.enable_cli Off Off > opcache.enable_file_override On On > opcache.error_log /var/log/php_error.log /var/log/php_error.log > opcache.fast_shutdown 1 1 > opcache.file_update_protection 2 2 > opcache.force_restart_timeout 180 180 > opcache.huge_code_pages On On > opcache.inherited_hack On On > opcache.interned_strings_buffer 8 8 > opcache.lockfile_path /tmp /tmp > opcache.log_verbosity_level 1 1 > opcache.max_accelerated_files 1000 1000 > opcache.max_file_size 327680 327680 > opcache.max_wasted_percentage 5 5 > opcache.memory_consumption 128 128 > opcache.opt_debug_level 0 0 > opcache.optimization_level 0x7FFFBFFF 0x7FFFBFFF > opcache.preferred_memory_model no value no value > opcache.protect_memory 0 0 > opcache.restrict_api /usr/share/php/zendoptimizer.php > /usr/share/php/zendoptimizer.php > opcache.revalidate_freq 5 5 > opcache.revalidate_path Off Off > opcache.save_comments 0 0 > opcache.use_cwd On On > opcache.validate_permission Off Off > opcache.validate_root Off Off > opcache.validate_timestamps On >
You have xdebug enabled...

lists@rhsoft.net

9 years ago
Am 02.05.2017 um 20:53 schrieb Nikita Popov:
> These results are very unlikely. I'm 95% sure your benchmark is > broken. My first guess would be that you're benchmarking PHP + > JIT against PHP without opcache. Please share the relevant > (opcache-related) portion of the php.inis you used > > no they are not - since i build RPM packages and even the whole > spec-file is unchanged, only the tarball changed and the build is > highly optimized i can assure you for 100% that i compare PHP > 7.1.5RC1 with https://github.com/zendtech/php-src > <https://github.com/zendtech/php-src> downloaded today > > maybe the PGO-profiling running autotests and fuzzy-calls on the > whole application as well as 2000 cms-requests combined with the > compiler flags improves the JIT itself > > without opcache the results for 7.1.5 are *dramatically* slower > > and i repeated the test upgrade/downgrade packages and run "ab" > multiple times on that machine - attached the "php.spec" which is > used for the build > > in just downloaded the zip from https://github.com/zendtech/php-src > <https://github.com/zendtech/php-src>, renamed it to "php-7.2.0", > made a tar.xz archive, changed the version on teh frist line in the > spec file and fired the build/profiling - nothing else changed > > You have xdebug enabled...
loaded, but not enabled xdebug.default_enable = 0 xdebug.profiler_enable = 0 xdebug.profiler_enable_trigger = 1 7.1.5: Requests per second: 136.46 7.1.5 opcache: Requests per second: 316.77 7.2.0 JIT PGO: Requests per second: 925.96 7.2.0 JIT NON-PGO: Requests per second: 849.99 around 8% are the difference with or without PGO, no idea how much strict-types and a 100% typehinted codebase makes a difference to the JIT operations

Nikita Popov

9 years ago
On Tue, May 2, 2017 at 9:04 PM, lists@rhsoft.net <lists@rhsoft.net> wrote:
> > Am 02.05.2017 um 20:53 schrieb Nikita Popov: > >> These results are very unlikely. I'm 95% sure your benchmark is >> broken. My first guess would be that you're benchmarking PHP + >> JIT against PHP without opcache. Please share the relevant >> (opcache-related) portion of the php.inis you used >> >> no they are not - since i build RPM packages and even the whole >> spec-file is unchanged, only the tarball changed and the build is >> highly optimized i can assure you for 100% that i compare PHP >> 7.1.5RC1 with https://github.com/zendtech/php-src >> <https://github.com/zendtech/php-src> downloaded today >> >> maybe the PGO-profiling running autotests and fuzzy-calls on the >> whole application as well as 2000 cms-requests combined with the >> compiler flags improves the JIT itself >> >> without opcache the results for 7.1.5 are *dramatically* slower >> >> and i repeated the test upgrade/downgrade packages and run "ab" >> multiple times on that machine - attached the "php.spec" which is >> used for the build >> >> in just downloaded the zip from https://github.com/zendtech/php-src >> <https://github.com/zendtech/php-src>, renamed it to "php-7.2.0", >> made a tar.xz archive, changed the version on teh frist line in the >> spec file and fired the build/profiling - nothing else changed >> > > > >> You have xdebug enabled... >> > > loaded, but not enabled > > xdebug.default_enable = 0 > xdebug.profiler_enable = 0 > xdebug.profiler_enable_trigger = 1 > > 7.1.5: Requests per second: 136.46 > 7.1.5 opcache: Requests per second: 316.77 > 7.2.0 JIT PGO: Requests per second: 925.96 > 7.2.0 JIT NON-PGO: Requests per second: 849.99 > > around 8% are the difference with or without PGO, no idea how much > strict-types and a 100% typehinted codebase makes a difference to the JIT > operations >
This is a common misconception. If you have loaded xdebug you will incur a major performance hit, regardless of ini settings. xdebug.default_enable is a badly named option, which controls display of stack traces, not whether xdebug is enabled. As far as I know, there is no way to disable xdebug once it has been loaded. Please unload xdebug and try again. Nikita

lists@rhsoft.net

9 years ago
Am 02.05.2017 um 21:12 schrieb Nikita Popov:
> On Tue, May 2, 2017 at 9:04 PM, lists@rhsoft.net > >> You have xdebug enabled... > > loaded, but not enabled > > xdebug.default_enable = 0 > xdebug.profiler_enable = 0 > xdebug.profiler_enable_trigger = 1 > > 7.1.5: Requests per second: 136.46 > 7.1.5 opcache: Requests per second: 316.77 > 7.2.0 JIT PGO: Requests per second: 925.96 > 7.2.0 JIT NON-PGO: Requests per second: 849.99 > > around 8% are the difference with or without PGO, no idea how much > strict-types and a 100% typehinted codebase makes a difference to > the JIT operations > > This is a common misconception. If you have loaded xdebug you will incur > a major performance hit, regardless of ini settings. > xdebug.default_enable is a badly named option, which controls display of > stack traces, not whether xdebug is enabled. As far as I know, there is > no way to disable xdebug once it has been loaded. > > Please unload xdebug and try again
argh - when the trigger is set to generate profiling files runtime is doubled compared with xdebug just loaded but indeed xdebug has a massive impact 7.1.5 PGO: Requests per second: 886.63 7.2.0 JIT PGO: Requests per second: 925.96 OK, than it are "only" 5% on a highly optimized codebase interesting is the impact of PGO-profiling which has likely a code-coverage nearly to 95% and that without xdebug our cms-system seems to be that efficient that it nearly runs as fast on a quad-core i7 from 2011 than on a 12-core Xeon from 2015.....

Sara Golemon

9 years ago
On Tue, May 2, 2017 at 2:51 PM, lists@rhsoft.net <lists@rhsoft.net> wrote:
> OK, than it are "only" 5% on a highly optimized codebase >
Awww... that makes me kinda sad to hear. But perhaps with more work that needle can be moved upwards. One question which I didn't see answered (in my admittedly quick scan): What codebase are you testing against? Wordpress stock install? -Sara

lists@rhsoft.net

9 years ago
Am 03.05.2017 um 02:37 schrieb Sara Golemon:
> On Tue, May 2, 2017 at 2:51 PM, lists@rhsoft.net <lists@rhsoft.net> wrote: >> OK, than it are "only" 5% on a highly optimized codebase >> > Awww... that makes me kinda sad to hear. But perhaps with more work > that needle can be moved upwards. > > One question which I didn't see answered (in my admittedly quick > scan): What codebase are you testing against? Wordpress stock > install?
100% internal codebase without 3rd party libraries developed over the last 15 years and in the meantime 100% strict-types/typehints/return-types only missing some commented nullable/void return types because i need to wait with 7.1.x for another busy guy with some of his instances :-) the profiling system is our internal demo-system with every depolyable module installed, runs the auto-testsuite inclduing refelection fuzzy-calls * override config and switch to a clean database * create 86 tables running the installer * 60 unit tests (heavy sound on a RAID10 HDD system) * 1200 reflection-fuzzy calls with random params to methdos * calling every backend url with a configured admin user * spider over 1600 frontend urls (curl, dom) * controlled by a bash/php mix running also the intermediate CLI that beast is highly optimized and the core-cms with just 4 navigation points and some lines of text has 12 function calls above 1% in xdebug :-) well, the demo-install for profiling has magnitudes more running code so 5% is not that bad - PHP 5.6 to 7.0 brought only 45% on that codebase while in the meantime the performance of the code got doubled at it's own (thanks xdebug) - hence someone heard my cry "WTF unbelievebale " after the first run not realize that the loaded xdebug without JIT makes most of the factor 2.5 difference while my epectation was 5-10% if any measureable improvement at all http://corecms.thelounge.net/show_content.php?sid=2 0.0027 / 0.0013 - cuurently running on 7.0.18 0.0027: Header always set "X-Response-Time" "%D us" 0.0013: microtime(true) first line versus microtime(true) final __________________________________ 3500 requests/second with 7.2-JIT on a 4-core i7 versus 4500/second with 7.8.18 on a 12-core Xeon tells me that PHP itself is only some part of the runtime of the ab-benchmark given that "ab" on my testmachine eats nearly one of the 4 cores at it's own and even under 100% load generate times in the browser stays below 0.01 seconds __________________________________ 250 sequentiell curl requests parsing the "X-Response-Time" average with identical content DYNAMIC CMS: 2382 us STATIC HTM: 242 us STATIC PHP: 288 us

Alex Sky

9 years ago
> 7.1.5 PGO: Requests per second: 886.63 > 7.2.0 JIT PGO: Requests per second: 925.96 > > OK, than it are "only" 5% on a highly optimized codebase
Try PHP script as daemon mode, will speed up 2x