Everything
php
Learn Community Releases The PHP Project
Home » Project » Newsgroups » php.internals » RFC: execution opcode file without php source code file »

RFC: execution opcode file without php source code file

php.internals

肖 鑫鑫

5 years ago
I commit a new request path, the request is about execution opcode file without php source code file this RFC provides similar to class file of java and pyo file of python. patch is: https://github.com/php/php-src/pull/6146

Michał Brzuchalski

5 years ago
Hi, czw., 1 paź 2020 o 11:36 肖 鑫鑫 <chopins.xiao@live.com> napisał(a):
> I commit a new request path, the request is about execution opcode file > without php source code file > this RFC provides similar to class file of java and pyo file of python. > patch is: https://github.com/php/php-src/pull/6146
I had exact the same feature in my mind and working implementation few years ago when PHP 7.0 appeared on horizon which didn't get a Zend Guard support. I like the idea a lot. One risk I see here is people putting whole framework or apps into one PHP file before calling `opcache_compile_file` to produce one big binary file. Would it be possible to move this feature to separate function and pass iterable list of files? I am not sure if with current OPCache file format it's possible but if that might reduce amount of steps required to produce one huge OPCache file which then can be loaded by interpreter. Second thing worth of noting is that PR examples use CLI SAPI and lacks of examples how FPM can load it. Would it require at least an index.php with `<?php include 'myapp.bin';` ? Cheers, Michał Marcin Brzuchalski

Henrik Skov

5 years ago
Hi list ! I would support such a feature as well. I am working on an implementation that would integrate PHAR support and opcode cache binary files (so that you could make a PHAR file that contains binary (opcode cache) files) I am voicing my opinion here because the current PHAR implementation does not work with binary (opcode cache) files. (The problem seems to be that the PHAR extension directly extracts/opens the phar file directory, compiles the file into opcodes and then executes them) - But compiling a file into opcodes that is already a binary (opcode cache) file yields garbage... So if this feature were to be implemented, the implementation would have to (in my opinion) solve this problem in the PHAR extension. The way I have worked around this problem is to implement some extract functions (require_once_bin(), require_bin(), include_bin() and include_once_bin() and then make sure that autoloaders used in inside the PHAR use those when reading from/including from that PHAR file that contains opcode binary files. /Henrik On 01-Oct-20 12:36, Michał Marcin Brzuchalski wrote:

Rowan Collins

5 years ago
Hi, On 1 October 2020 10:36:20 BST, "肖 鑫鑫" <chopins.xiao@live.com> wrote:
>I commit a new request path, the request is about execution opcode file >without php source code file >this RFC provides similar to class file of java and pyo file of python. >patch is: https://github.com/php/php-src/pull/6146
I'm sure someone who knows the internals better can clarify, but my understanding is that PHP OpCodes don't currently have any stability guarantee, so you can't rely on a binary taken from one version will run on another, even within a release. In order to be useful, this will therefore need two things: - a header in the file identifying the engine version it was compiled for, with an error raised on any mismatch - a policy of how to manage that version number, and how users can know which versions their binary files will work on There's probably a limit to how stable we can (or want to) make the VM, so these files will never be as portable as a Java class file or .Net assembly. Regards,
-- Rowan Tommins [IMSoP]

Nikita Popov

5 years ago
On Thu, Oct 1, 2020 at 1:44 PM Rowan Tommins <rowan.collins@gmail.com> wrote:
> Hi, > > On 1 October 2020 10:36:20 BST, "肖 鑫鑫" <chopins.xiao@live.com> wrote: > >I commit a new request path, the request is about execution opcode file > >without php source code file > >this RFC provides similar to class file of java and pyo file of python. > >patch is: https://github.com/php/php-src/pull/6146 > > I'm sure someone who knows the internals better can clarify, but my > understanding is that PHP OpCodes don't currently have any stability > guarantee, so you can't rely on a binary taken from one version will run on > another, even within a release. > > In order to be useful, this will therefore need two things: > > - a header in the file identifying the engine version it was compiled for, > with an error raised on any mismatch > - a policy of how to manage that version number, and how users can know > which versions their binary files will work on > > There's probably a limit to how stable we can (or want to) make the VM, so > these files will never be as portable as a Java class file or .Net assembly. >
This is correct. PHP uses the "system ID" to determine whether it is possible to reuse compiled opcodes. This system ID is used by the existing file cache. The linked pull request however explicitly disables this check, which is not safe. And with that check enabled, this feature would probably have limited usefulness, as the cached file would be bound to a specific PHP patch release (and to some degree to which extensions are loaded). It would be good to know what the actual use-case for this is. If the goal here is to distribute pre-compiled PHP code without the source code, then this is not going to work without making the opcode format stable (which, I think, is pretty unlikely.) If the goal is improving cold startup performance, then I wonder what the benefit over the existing file cache is. Regards, Nikita

Dik Takken

5 years ago
On 01-10-2020 14:30, Nikita Popov wrote:
> It would be good to know what the actual use-case for this is. If the goal > here is to distribute pre-compiled PHP code without the source code, then > this is not going to work without making the opcode format stable (which, I > think, is pretty unlikely.) If the goal is improving cold startup > performance, then I wonder what the benefit over the existing file cache is.
The only use case I see is to package commercial closed source applications as Docker containers. That allows the packager to ship the compiled code along with the exact PHP version and configuration that can run it reliably. Regards, Dik Takken

Unnamed Person

5 years ago
On Thu, Oct 1, 2020 at 9:14 AM Dik Takken <dik.takken@gmail.com> wrote:
> > On 01-10-2020 14:30, Nikita Popov wrote: > > It would be good to know what the actual use-case for this is. If the goal > > here is to distribute pre-compiled PHP code without the source code, then > > this is not going to work without making the opcode format stable (which, I > > think, is pretty unlikely.) If the goal is improving cold startup > > performance, then I wonder what the benefit over the existing file cache is. > > The only use case I see is to package commercial closed source > applications as Docker containers. That allows the packager to ship the > compiled code along with the exact PHP version and configuration that > can run it reliably. > > Regards, > Dik Takken
Note that for PHP 8.0 we made the system ID change even more than it used to because doing so fixes sigsegvs when switching what extensions are loaded (common when trialing an APM product, or loading and unloading xdebug) when file cache is used. This means that it is even more important to have sources than in the past, and necessarily so because of real segmentation faults. So I second Nikita's opinion: this is not going to work well without stabilizing the opcode format. This may be possible, but it would be a lot of work; it's not happening soon. If you do it anyway be prepared for plenty of crash complaints.

Pierre

5 years ago
Le 01/10/2020 à 18:32, Levi Morrison via internals a écrit :
> Note that for PHP 8.0 we made the system ID change even more than it > used to because doing so fixes sigsegvs when switching what extensions > are loaded (common when trialing an APM product, or loading and > unloading xdebug) when file cache is used. This means that it is even > more important to have sources than in the past, and necessarily so > because of real segmentation faults. > > So I second Nikita's opinion: this is not going to work well without > stabilizing the opcode format. This may be possible, but it would be a > lot of work; it's not happening soon. If you do it anyway be prepared > for plenty of crash complaints. >
Hello, I know this may sound naive, but would it be a bad idea to stabilize opcodes (which probably is much harder than it looks), having the VM being built following a thorough specification ? I could open the door to many things, such as third party compilers or optimizers ? I don't know much Zend VM internals, but having the possibility to have PHP binaries other than phar sound compelling to me, I wish I could deliver upgrades using a single binary instead of a 100k PHP files delivery, especially in docker environments. It may also open the door to autoload-less execution, link time-like optimizations, and even open a slight door toward reflection around modules/packages ? Regards, Pierre

Rowan Collins

5 years ago
On Thu, 1 Oct 2020 at 16:13, Dik Takken <dik.takken@gmail.com> wrote:
> The only use case I see is to package commercial closed source > applications as Docker containers. That allows the packager to ship the > compiled code along with the exact PHP version and configuration that > can run it reliably. >
Could this not be achieved by freezing the existing OpCache file cache into the Docker image and telling OpCache not to validate timestamps, so that it always loads from the cache? That way, your application wouldn't even need to be modified, as the require_once lines would still refer to the original source paths. Regards,
-- Rowan Tommins [IMSoP]

David Rodrigues

5 years ago
Hello folks, Instead of an opcode without a php source file, that I imagine is to protect the code itself, why not a method to encrypt phar files (not like a password). I do not know if exists a secure method to decrypt to execute only, without reveals the original source code, but maybe it could be done. So opcode could be generated based on encrypted phar to give more speed. Atenciosamente, David Rodrigues Em sex., 2 de out. de 2020 às 13:52, Rowan Tommins <rowan.collins@gmail.com> escreveu:

Walter Parker

5 years ago
On Fri, Oct 2, 2020 at 10:08 AM David Rodrigues <david.proweb@gmail.com> wrote:
> > Hello folks, > > Instead of an opcode without a php source file, that I imagine is to > protect the code itself, why not a method to encrypt phar files (not like a > password). I do not know if exists a secure method to decrypt to execute > only, without reveals the original source code, but maybe it could be done. > So opcode could be generated based on encrypted phar to give more speed. > >
The flaw with decryption is that the image will have to get the decryption key from somewhere. The history of the DVD industry over the past 25 years has shown that getting that key to an untrusted device and then keeping it secret is much harder than it looks (when people care enough to go after the key, even hardware locks can be made to fail). Or look at the phone industry, even phones with hardware & software locked crypto get broken open. Note, if the device is trusted, then encryption and decryption are not needed (as you would then be able to trust the device to not give out the source code). Also, note that reverse engineering (decompling) raw binaries is a thing (there are tools to do that). You need to decide on you threat model for your code to decide on how much it is worth to lock your code. Walter
-- The greatest dangers to liberty lurk in insidious encroachment by men of zeal, well-meaning but without understanding. -- Justice Louis D. Brandeis