Everything
php
Learn Community Releases The PHP Project
Home » Project » Newsgroups » php.internals » HYBRID VM »

HYBRID VM

php.internals

Dmitry Stogov

9 years ago
Hi, Recently, I committed HYBRID VM into master, but didn't enable it by default yet. It provides significant performance improvement on small benchmarks (1.5 times faster on bench.php) and slight improvement on real-life apps (1-2% on wordpress). Currently it improves PHP only on x86, x86_64 and PPC compiled with GCC. Now, after additional testing, I like to make HYBRID VM the default VM for PHP-7.2 and above. I would glad to do this without RFC, because it's a self containing change that doesn't affect anything except the VM itself. It may only break some "system" PHP extensions (debuggers, profilers), that relay on CALL VM implementation. phpdbg and xdebug are not affected and work out of the box. Anyone may regenerate and test it. php zend_vm_gen.php --with-vm-kind=HYBRID make Thanks. Dmitry.

lists@rhsoft.net

9 years ago
Am 15.05.2017 um 16:43 schrieb Dmitry Stogov:
> php zend_vm_gen.php --with-vm-kind=HYBRID > > make
shouldn't that be a ./configure option? calls like "php zend_vm_gen.php" are not self contained in a rpmbuild

Sara Golemon

9 years ago
On Mon, May 15, 2017 at 9:51 AM, lists@rhsoft.net <lists@rhsoft.net> wrote:
> Am 15.05.2017 um 16:43 schrieb Dmitry Stogov: >> >> php zend_vm_gen.php --with-vm-kind=HYBRID >> > shouldn't that be a ./configure option? > calls like "php zend_vm_gen.php" are not self contained in a rpmbuild >
atm, no. The VM gen is (and has been since 5.0) a development time generator which is checked into GIT (because it requires PHP to be built to run, and must run before PHP is built). rpmbuilders (probably) never need worry about selecting alternate VMs, but if they do there must be a means to patch codebases before building (distros do this routinely), hooking into that should be a possibility. That said, the only argument against adding it as a ./configure option is that it's a bootstrap step (as mentioned above). I suppose we could use the checked in version of the VM when the option isn't specified, and error if one tried to specify the option without having a PHP version already around. That's orthogonal to this thread, however. Dmitry; You've got my thumbs up. No need for an RFC for something purely internal like this. -Sara

lists@rhsoft.net

9 years ago
Am 15.05.2017 um 17:30 schrieb Sara Golemon:
> On Mon, May 15, 2017 at 9:51 AM, lists@rhsoft.net <lists@rhsoft.net> wrote: >> Am 15.05.2017 um 16:43 schrieb Dmitry Stogov: >>> >>> php zend_vm_gen.php --with-vm-kind=HYBRID >>> >> shouldn't that be a ./configure option? >> calls like "php zend_vm_gen.php" are not self contained in a rpmbuild >> > atm, no. The VM gen is (and has been since 5.0) a development time > generator which is checked into GIT (because it requires PHP to be > built to run, and must run before PHP is built). rpmbuilders > (probably) never need worry about selecting alternate VMs, but if they > do there must be a means to patch codebases before building (distros > do this routinely), hooking into that should be a possibility.
rpmbuilders are not only distributions people which optimize PHP for their hardware and with PGO profiling (like me) and anybody else building from source does himself a favour when build packages i looked at that script and found also ZEND_VM_KIND_GOTO not sure what and if that has to to with --enable-re2c-cgoto which is enabled here
> That said, the only argument against adding it as a ./configure option > is that it's a bootstrap step (as mentioned above). I suppose we > could use the checked in version of the VM when the option isn't > specified, and error if one tried to specify the option without having > a PHP version already around. > > That's orthogonal to this thread, however. > > Dmitry; You've got my thumbs up. No need for an RFC for something > purely internal like this.
ok

Derick Rethans

9 years ago
Hi! On Mon, 15 May 2017, Dmitry Stogov wrote:
> Recently, I committed HYBRID VM into master, but didn't enable it by > default yet. > > It provides significant performance improvement on small benchmarks > (1.5 times faster on bench.php) and slight improvement on real-life > apps (1-2% on wordpress). Currently it improves PHP only on x86, > x86_64 and PPC compiled with GCC.
It also crashes instantly when Xdebug is loaded. I think we should resolve that before making it default for 7.2 and above.
> > Now, after additional testing, I like to make HYBRID VM the default VM for PHP-7.2 and above. > > I would glad to do this without RFC, because it's a self containing change that doesn't affect anything except the VM itself. > > It may only break some "system" PHP extensions (debuggers, profilers), that relay on CALL VM implementation. > > phpdbg and xdebug are not affected and work out of the box.
Uh? Definitely just segfaults here: Number of tests : 476 388 Tests skipped : 88 ( 18.5%) -------- Tests warned : 0 ( 0.0%) ( 0.0%) Tests failed : 384 ( 80.7%) ( 99.0%) Expected fail : 0 ( 0.0%) ( 0.0%) Tests passed : 4 ( 0.8%) ( 1.0%) derick@singlemalt:~/dev/php/xdebug-xdebug $ cat tests/001.php <?php echo "xdebug extension is available"; ?> [GIT: master][PHP: 7.2.0-dev ] derick@singlemalt:~/dev/php/xdebug-xdebug $ gdb --args /usr/local/php/trunk/bin/php -n -c '/home/derick/dev/php/xdebug-xdebug/tmp-php.ini' -d "output_handler=" -d "open_basedir=" -d "disable_functions=" -d "output_buffering=Off" -d "error_reporting=32767" -d "display_errors=1" -d "display_startup_errors=1" -d "log_errors=0" -d "html_errors=0" -d "track_errors=0" -d "report_memleaks=1" -d "report_zend_debug=0" -d "docref_root=" -d "docref_ext=.html" -d "error_prepend_string=" -d "error_append_string=" -d "auto_prepend_file=" -d "auto_append_file=" -d "ignore_repeated_errors=0" -d "precision=14" -d "memory_limit=128M" -d "log_errors_max_len=0" -d "opcache.fast_shutdown=0" -d "opcache.file_update_protection=0" -d "extension_dir=/home/derick/dev/php/xdebug-xdebug/modules/" -d "zend_extension=/home/derick/dev/php/xdebug-xdebug/modules/xdebug.so" -d "session.auto_start=0" -d "zlib.output_compression=Off" -d "xdebug.default_enable=0" -d "mbstring.func_overload=0" -d "xdebug.profiler_enable=0" -f "/home/derick/dev/php/xdebug-xdebug/tests/001.php" GNU gdb (Debian 7.12-6) 7.12.0.20161007-git Copyright (C) 2016 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html> This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. Type "show copying" and "show warranty" for details. This GDB was configured as "x86_64-linux-gnu". Type "show configuration" for configuration details. For bug reporting instructions, please see: <http://www.gnu.org/software/gdb/bugs/>. Find the GDB manual and other documentation resources online at: <http://www.gnu.org/software/gdb/documentation/>. For help, type "help". Type "apropos word" to search for commands related to "word"... Reading symbols from /usr/local/php/trunk/bin/php...done. (gdb) run Starting program: /usr/local/php/trunk/bin/php -n -c /home/derick/dev/php/xdebug-xdebug/tmp-php.ini -d output_handler= -d open_basedir= -d disable_functions= -d output_buffering=Off -d error_reporting=32767 -d display_errors=1 -d display_startup_errors=1 -d log_errors=0 -d html_errors=0 -d track_errors=0 -d report_memleaks=1 -d report_zend_debug=0 -d docref_root= -d docref_ext=.html -d error_prepend_string= -d error_append_string= -d auto_prepend_file= -d auto_append_file= -d ignore_repeated_errors=0 -d precision=14 -d memory_limit=128M -d log_errors_max_len=0 -d opcache.fast_shutdown=0 -d opcache.file_update_protection=0 -d extension_dir=/home/derick/dev/php/xdebug-xdebug/modules/ -d zend_extension=/home/derick/dev/php/xdebug-xdebug/modules/xdebug.so -d session.auto_start=0 -d zlib.output_compression=Off -d xdebug.default_enable=0 -d mbstring.func_overload=0 -d xdebug.profiler_enable=0 -f /home/derick/dev/php/xdebug-xdebug/tests/001.php [Thread debugging using libthread_db enabled] Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1". xdebug extension is available Program received signal SIGSEGV, Segmentation fault. 0x0000555555e4c8f9 in execute_ex (ex=0x555555dd02ff <ZEND_USER_OPCODE_SPEC_HANDLER+271>) at /home/derick/dev/php/php-src.git/Zend/zend_vm_execute.h:59065 59065 HYBRID_BREAK(); (gdb) bt #0 0x0000555555e4c8f9 in execute_ex (ex=0x555555dd02ff <ZEND_USER_OPCODE_SPEC_HANDLER+271>) at /home/derick/dev/php/php-src.git/Zend/zend_vm_execute.h:59065 #1 0x0000555555e4c8f6 in execute_ex (ex=0x555555dd02ff <ZEND_USER_OPCODE_SPEC_HANDLER+271>) at /home/derick/dev/php/php-src.git/Zend/zend_vm_execute.h:59064 #2 0x0000555555e4c8f6 in execute_ex (ex=0x7ffff1a1f030) at /home/derick/dev/php/php-src.git/Zend/zend_vm_execute.h:59064 #3 0x00007ffff17be907 in xdebug_execute_ex (execute_data=0x7ffff1a1f030) at /home/derick/dev/php/xdebug-xdebug/xdebug.c:1795 #4 0x0000555555e51a1d in zend_execute (op_array=0x7ffff1a83300, return_value=0x0) at /home/derick/dev/php/php-src.git/Zend/zend_vm_execute.h:62949 #5 0x0000555555d6d7f6 in zend_execute_scripts (type=8, retval=0x0, file_count=3) at /home/derick/dev/php/php-src.git/Zend/zend.c:1537 #6 0x0000555555cdbd82 in php_execute_script (primary_file=0x7fffffffd7c0) at /home/derick/dev/php/php-src.git/main/main.c:2548 #7 0x0000555555e5408e in do_cli (argc=68, argv=0x5555569c5e70) at /home/derick/dev/php/php-src.git/sapi/cli/php_cli.c:997 #8 0x0000555555e55048 in main (argc=68, argv=0x5555569c5e70) at /home/derick/dev/php/php-src.git/sapi/cli/php_cli.c:1390 (gdb) Latest Xdebug from GitHub. cheers, Derick
-- https://derickrethans.nl | https://xdebug.org | https://dram.io Like Xdebug? Consider a donation: https://xdebug.org/donate.php twitter: @derickr and @xdebug

Dmitry Stogov

9 years ago
Hi Derick, Sorry, it seems I tested it improperly. Anyway, I found and fixed the problem. It was a bug in USER_OPCODE handling in HYBRID VM. Thanks. Dmitry. ________________________________ From: Derick Rethans <derick@php.net> Sent: Monday, May 15, 2017 6:32:19 PM To: Dmitry Stogov Cc: PHP internals list; Nikita Popov; Xinchen Hui; Sara Golemon; Zeev Suraski Subject: Re: [PHP-DEV] HYBRID VM Hi! On Mon, 15 May 2017, Dmitry Stogov wrote:
> Recently, I committed HYBRID VM into master, but didn't enable it by > default yet. > > It provides significant performance improvement on small benchmarks > (1.5 times faster on bench.php) and slight improvement on real-life > apps (1-2% on wordpress). Currently it improves PHP only on x86, > x86_64 and PPC compiled with GCC.
It also crashes instantly when Xdebug is loaded. I think we should resolve that before making it default for 7.2 and above.
> > Now, after additional testing, I like to make HYBRID VM the default VM for PHP-7.2 and above. > > I would glad to do this without RFC, because it's a self containing change that doesn't affect anything except the VM itself. > > It may only break some "system" PHP extensions (debuggers, profilers), that relay on CALL VM implementation. > > phpdbg and xdebug are not affected and work out of the box.
Uh? Definitely just segfaults here: Number of tests : 476 388 Tests skipped : 88 ( 18.5%) -------- Tests warned : 0 ( 0.0%) ( 0.0%) Tests failed : 384 ( 80.7%) ( 99.0%) Expected fail : 0 ( 0.0%) ( 0.0%) Tests passed : 4 ( 0.8%) ( 1.0%) derick@singlemalt:~/dev/php/xdebug-xdebug $ cat tests/001.php <?php echo "xdebug extension is available"; ?> [GIT: master][PHP: 7.2.0-dev ] derick@singlemalt:~/dev/php/xdebug-xdebug $ gdb --args /usr/local/php/trunk/bin/php -n -c '/home/derick/dev/php/xdebug-xdebug/tmp-php.ini' -d "output_handler=" -d "open_basedir=" -d "disable_functions=" -d "output_buffering=Off" -d "error_reporting=32767" -d "display_errors=1" -d "display_startup_errors=1" -d "log_errors=0" -d "html_errors=0" -d "track_errors=0" -d "report_memleaks=1" -d "report_zend_debug=0" -d "docref_root=" -d "docref_ext=.html" -d "error_prepend_string=" -d "error_append_string=" -d "auto_prepend_file=" -d "auto_append_file=" -d "ignore_repeated_errors=0" -d "precision=14" -d "memory_limit=128M" -d "log_errors_max_len=0" -d "opcache.fast_shutdown=0" -d "opcache.file_update_protection=0" -d "extension_dir=/home/derick/dev/php/xdebug-xdebug/modules/" -d "zend_extension=/home/derick/dev/php/xdebug-xdebug/modules/xdebug.so" -d "session.auto_start=0" -d "zlib.output_compression=Off" -d "xdebug.default_enable=0" -d "mbstring.func_overload=0" -d "xdebug.profiler_enable=0" -f "/home/derick/dev/php/xdebug-xdebug/tests/001.php" GNU gdb (Debian 7.12-6) 7.12.0.20161007-git Copyright (C) 2016 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html> This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. Type "show copying" and "show warranty" for details. This GDB was configured as "x86_64-linux-gnu". Type "show configuration" for configuration details. For bug reporting instructions, please see: <http://www.gnu.org/software/gdb/bugs/>. Find the GDB manual and other documentation resources online at: <http://www.gnu.org/software/gdb/documentation/>. For help, type "help". Type "apropos word" to search for commands related to "word"... Reading symbols from /usr/local/php/trunk/bin/php...done. (gdb) run Starting program: /usr/local/php/trunk/bin/php -n -c /home/derick/dev/php/xdebug-xdebug/tmp-php.ini -d output_handler= -d open_basedir= -d disable_functions= -d output_buffering=Off -d error_reporting=32767 -d display_errors=1 -d display_startup_errors=1 -d log_errors=0 -d html_errors=0 -d track_errors=0 -d report_memleaks=1 -d report_zend_debug=0 -d docref_root= -d docref_ext=.html -d error_prepend_string= -d error_append_string= -d auto_prepend_file= -d auto_append_file= -d ignore_repeated_errors=0 -d precision=14 -d memory_limit=128M -d log_errors_max_len=0 -d opcache.fast_shutdown=0 -d opcache.file_update_protection=0 -d extension_dir=/home/derick/dev/php/xdebug-xdebug/modules/ -d zend_extension=/home/derick/dev/php/xdebug-xdebug/modules/xdebug.so -d session.auto_start=0 -d zlib.output_compression=Off -d xdebug.default_enable=0 -d mbstring.func_overload=0 -d xdebug.profiler_enable=0 -f /home/derick/dev/php/xdebug-xdebug/tests/001.php [Thread debugging using libthread_db enabled] Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1". xdebug extension is available Program received signal SIGSEGV, Segmentation fault. 0x0000555555e4c8f9 in execute_ex (ex=0x555555dd02ff <ZEND_USER_OPCODE_SPEC_HANDLER+271>) at /home/derick/dev/php/php-src.git/Zend/zend_vm_execute.h:59065 59065 HYBRID_BREAK(); (gdb) bt #0 0x0000555555e4c8f9 in execute_ex (ex=0x555555dd02ff <ZEND_USER_OPCODE_SPEC_HANDLER+271>) at /home/derick/dev/php/php-src.git/Zend/zend_vm_execute.h:59065 #1 0x0000555555e4c8f6 in execute_ex (ex=0x555555dd02ff <ZEND_USER_OPCODE_SPEC_HANDLER+271>) at /home/derick/dev/php/php-src.git/Zend/zend_vm_execute.h:59064 #2 0x0000555555e4c8f6 in execute_ex (ex=0x7ffff1a1f030) at /home/derick/dev/php/php-src.git/Zend/zend_vm_execute.h:59064 #3 0x00007ffff17be907 in xdebug_execute_ex (execute_data=0x7ffff1a1f030) at /home/derick/dev/php/xdebug-xdebug/xdebug.c:1795 #4 0x0000555555e51a1d in zend_execute (op_array=0x7ffff1a83300, return_value=0x0) at /home/derick/dev/php/php-src.git/Zend/zend_vm_execute.h:62949 #5 0x0000555555d6d7f6 in zend_execute_scripts (type=8, retval=0x0, file_count=3) at /home/derick/dev/php/php-src.git/Zend/zend.c:1537 #6 0x0000555555cdbd82 in php_execute_script (primary_file=0x7fffffffd7c0) at /home/derick/dev/php/php-src.git/main/main.c:2548 #7 0x0000555555e5408e in do_cli (argc=68, argv=0x5555569c5e70) at /home/derick/dev/php/php-src.git/sapi/cli/php_cli.c:997 #8 0x0000555555e55048 in main (argc=68, argv=0x5555569c5e70) at /home/derick/dev/php/php-src.git/sapi/cli/php_cli.c:1390 (gdb) Latest Xdebug from GitHub. cheers, Derick
-- https://derickrethans.nl | https://xdebug.org | https://dram.io Like Xdebug? Consider a donation: https://xdebug.org/donate.php twitter: @derickr and @xdebug

lists@rhsoft.net

9 years ago
Am 15.05.2017 um 16:43 schrieb Dmitry Stogov:
> Hi, > > > Recently, I committed HYBRID VM into master, but didn't enable it by default yet. > > It provides significant performance improvement on small benchmarks (1.5 times faster on bench.php) and slight improvement on real-life apps (1-2% on wordpress). Currently it improves PHP only on x86, x86_64 and PPC compiled with GCC. > > Anyone may regenerate and test it. > > > php zend_vm_gen.php --with-vm-kind=HYBRID > > make
benchmarks for our inhouse CMS on my home-machine * Intel(R) Core(TM) i7-3770 CPU @ 3.40GHz * PGO-Build -mtune=sandybridge for our application there is real difference, just look at the two different runs with the hybrid-vm - which is at least not bad at the end "corecms" = a completly stripped own instance of our inhouse cms "contentlounge" = a expensive "generic formmailer" and other modules on the page the machine runs in a room with a high temperature and i faced larger differences even with the same build and nothing changed, maybe other applications will see more speed improvement, the codebase is highly optimized down to OPCODEs looking at xdebug-profiles and probably the most tiny codebase depending on the configured modules out there given a total LOC around 250000 lines in theory (but that includes also admin baclaneds for all sort of servers....) _______________________________________________________________ /usr/bin/php Zend/zend_vm_gen.php --with-vm-kind=CALL Server Hostname: corecms Concurrency Level: 50 Time taken for tests: 30.158 seconds Complete requests: 100000 Failed requests: 0 Total transferred: 383563326 bytes HTML transferred: 346464592 bytes Requests per second: 3315.84 [#/sec] (mean) Time per request: 15.079 [ms] (mean) Time per request: 0.302 [ms] (mean, across all concurrent requests) Transfer rate: 12420.25 [Kbytes/sec] received Server Hostname: contentlounge Concurrency Level: 50 Time taken for tests: 109.291 seconds Complete requests: 100000 Failed requests: 0 Total transferred: 4806868605 bytes HTML transferred: 4758197006 bytes Requests per second: 914.99 [#/sec] (mean) Time per request: 54.645 [ms] (mean) Time per request: 1.093 [ms] (mean, across all concurrent requests) Transfer rate: 42951.58 [Kbytes/sec] received _______________________________________________________________ /usr/bin/php Zend/zend_vm_gen.php --with-vm-kind=HYBRID Server Hostname: corecms Concurrency Level: 50 Time taken for tests: 31.528 seconds Complete requests: 100000 Failed requests: 0 Total transferred: 383561944 bytes HTML transferred: 346463930 bytes Requests per second: 3171.77 [#/sec] (mean) Time per request: 15.764 [ms] (mean) Time per request: 0.315 [ms] (mean, across all concurrent requests) Transfer rate: 11880.57 [Kbytes/sec] received Server Hostname: corecms Concurrency Level: 50 Time taken for tests: 31.248 seconds Complete requests: 100000 Failed requests: 0 Total transferred: 383561400 bytes HTML transferred: 346462104 bytes Requests per second: 3200.20 [#/sec] (mean) Time per request: 15.624 [ms] (mean) Time per request: 0.312 [ms] (mean, across all concurrent requests) Transfer rate: 11987.03 [Kbytes/sec] received Server Hostname: contentlounge Concurrency Level: 50 Time taken for tests: 106.127 seconds Complete requests: 100000 Failed requests: 0 Total transferred: 4806870511 bytes HTML transferred: 4758196867 bytes Requests per second: 942.27 [#/sec] (mean) Time per request: 53.063 [ms] (mean) Time per request: 1.061 [ms] (mean, across all concurrent requests) Transfer rate: 44232.14 [Kbytes/sec] received Server Hostname: contentlounge Concurrency Level: 50 Time taken for tests: 107.479 seconds Complete requests: 100000 Failed requests: 0 Total transferred: 4806867967 bytes HTML transferred: 4758194770 bytes Requests per second: 930.41 [#/sec] (mean) Time per request: 53.739 [ms] (mean) Time per request: 1.075 [ms] (mean, across all concurrent requests) Transfer rate: 43675.61 [Kbytes/sec] received _______________________________________________________________ for rpmbuilders: %prep export LANG=C %setup -q -n php-%{version} %patch1 -p1 -b .realpath %if "%{version}" >= "7.1.0" %patch3 -p1 -b .systzdata-71 %else %patch2 -p1 -b .systzdata-70 %endif # rebuild 'data_file.c' from current system libmagic starting with PHP 7.2 and when /usr/bin/php is installed %if "%{version}" >= "7.2.0" if [ -f /usr/bin/php ]; then /usr/bin/php ext/fileinfo/create_data_file.php /usr/share/misc/magic.mgc > ext/fileinfo/data_file.c fi %endif # generate zend-vm when /usr/bin/php is installed %if "%{version}" >= "7.2.0" if [ -f /usr/bin/php ]; then /usr/bin/php Zend/zend_vm_gen.php --with-vm-kind=HYBRID fi %else if [ -f /usr/bin/php ]; then /usr/bin/php Zend/zend_vm_gen.php --with-vm-kind=CALL fi %endif