RFC: Storing additional information for op_arrays

Derick Rethans schrieb:

Well the "official" way to use the reserved area of an opcode array is to get an handle from the zend engine for reserved space. This is done by ZEND_API int zend_get_resource_handle(zend_extension *extension) it returns an index you can simply use. (I think meanwhile APC will honor this and will not overwrite the first bytes).

The basic idea would be to add API functions like zend_op_array_add_data(key, data, size, flags) zend_op_array_remove_data()... zend_op_array_get_data, zend_op_array_get_all_data In the flags one could choose: Needs to be cached etc... Well and this will be stored in a datastructure that will just be pointed at by the op_array struct. So op_arrays will not really grow (just a single pointer added) and the whole overhead is only in the extensions that actually require such extra data and in the opcode caches that need to cache this. And I think the opcode cachers would just need to traverse the list of all data and store it with the op_array. I doubt that would take much changed in the cacher software. And well if the cache authors speak up, PHP can pretty much implement all the API functions required for such a step... Stefan Esser

Morning,

Yes it was a bug in APC. But the problem is that I doubt it is clearly documented how to get access to the "reserved resourced". Actually I believe Suhosin was the first open source solution that used these resources and the only closed source software that did it (and dit it long before Suhosin) are Zend tools.

Well the problem I want to be solving is the following 1) Extension generated extra data that is linked to the op_array (f.e. information like: was generated by eval()'d code, does use variable XYZ, uses session functionality, ....) 2) Naive way is to reserve a slot and simply put a pointer in there 3) Naive way works when there is no opcode cache, because then the extra data has to be regenerated everytime anyway (SLOW!) 4) Naive way works when the opcode cache does only work on a per process/thread basis, because then the pointer stays valid 5) Naive way FAILS when opcode cache stores data on disk and loads it back (pointer wrong, or process never generated the extra data) 6) Naive way FAILS when opcode cache shares data between processes (pointer wrong, ...) Now the idea was to create a solution for this problem that is compatible with all opcode cachers that follow some rules. Correct me if I am wrong, but I believed that op_arrays in shared memory cannot be used directly and usually have to be copied into the own address space to work on... If this is true then maybe the simplest idea would be to have some kind of suspend and resume functionality for op_arrays. When you put the op_array into the shared memory you call a suspend hook() extensions can register and when you read it from shared memory you call the resume() hook. This would be the simplest way, because then the extension could maybe clear its pointer in the extra slot so that it is NULL for the other process and then it knows inside the other process that it has to regenerate it for the other process. Or it simply regenerates the information from within the resume_hook(). I think this idea is pretty simple to implement. But my original idea was a little bit more complicated. I wanted that the suspend hook creates a binary data package (by extension) and that when the opcode cacher caches the binary data package it copies the data of the suspend hook into shared memory/writes it onto disk. On the other hand the resume hook will decode the extra data... To make it simple for the opcode cachers PHP could internally handle the whole combine data of extensions and split up data by extension stuff. So for opcode cachers they simply have to: 1) zend_suspend_op_array(op_array, *buffer, *size); 2) Store the extra data in buffer of size bytes together with the op_array 3) zend_resume_op_array(op_array, buffer, size); I know that there are some op_array() hooks for zend_extensions but I believe right now there is no way to hook "I just came from some kind of shared memory (memory/disk/network)" please fix me up so that I still work in your process.... Stefan Esser