Newcomer, PHP pre-processing, Magic Constants __FILE__ and __LINE__ override with #line like in C

php.internals

Ludovic Pouzenc

3 years ago
Hi, I'm a newcomer here. I came from https://wiki.php.net/rfc/howto. *TL;DR:* I want to know if a proposal of a #line parsing to alter __FILE__ and __LINE__ in PHP like it exists in C could be a good idea. I've searched about __LINE__ and __FILE__ in mailing archives and wiki, I don't really find things matching. I hope it's not because "__" are stripped before search. "Magic constants" didn't help neither. Let me know if I am rethinking and old thing already imagined here in the past. *Background / use case:* I think at first about webservices, maybe the useful also with browser/human-facing use cases. With PHP, it's hard to get high responsiveness under load if user code start by loading a composer's autoload.php, then (lazy) loads a whole framework with 10+ dependencies, then execute 1 SQL request, 1 json_encode() then output and exit. Even with "Slim" framework that should be minimal, using only the route feature, I see "strace" counting 200+ file-related syscalls per request with recent linux/debian/apache/php-fpm. It's not so hard to get it better with other langages : when they are compiled, or when framework loading happens at server start and to at each request. Web projects have more and more a "compilation" phase for minimising JS, for proprocessing scss to css, it is possible to hook it some PHP processing too. In my university, I started using a purpose-written PHP Preprocessor to minimize runtime dependency loading. It only understands #include like in C. From a src/mywebservice/v1/some-endpoint.php it will generate and build/mywebservice/v1/some-endpoint.php with all #include "some/path/somedep.php" replaced by the file's contents. So the generated some-endpoint.php have no run-time dependency at all: no autoload, no include(), no require(). I think it evens maximize the gains with APC caching. The preprocessor also generate things like #line 2 "some/path/somedep.php" where an #include was encountered, then a thing like #line 47 "src/mywebservice/v1/some-endpoint.php" right after the end of the inclusion. In C, a great concrete example of #line importance is working with a flex/bison parser generator. If PHP parser interpret #line as in C, __FILE__ and __LINE__ Magic Constants will be changed to source file and line, instead of generated file and line. It could greatly improve development write-then-rerun cycle. (missing ";" at line NN , other PHP Errors, Exception details/traces) I hope it could unlock many use cases where "big" PHP frameworks get really hard time to try to compete with other languages equivalents. Do you think it's an idea that is suitable to discuss, improve and submit as an RFC ? Regards,
-- Ludovic Pouzenc www.pouzenc.fr

Stephen Reay

3 years ago
> On 23 Oct 2022, at 20:43, Ludovic Pouzenc <ludovic@pouzenc.fr> wrote: > > Hi, > > I'm a newcomer here. I came from https://wiki.php.net/rfc/howto. > > *TL;DR:* I want to know if a proposal of a #line parsing to alter __FILE__ and __LINE__ in PHP like it exists in C could be a good idea. > > I've searched about __LINE__ and __FILE__ in mailing archives and wiki, I don't really find things matching. I hope it's not because "__" are stripped before search. "Magic constants" didn't help neither. Let me know if I am rethinking and old thing already imagined here in the past. > > *Background / use case:* I think at first about webservices, maybe the useful also with browser/human-facing use cases. > > With PHP, it's hard to get high responsiveness under load if user code start by loading a composer's autoload.php, then (lazy) loads a whole framework with 10+ dependencies, then execute 1 SQL request, 1 json_encode() then output and exit. Even with "Slim" framework that should be minimal, using only the route feature, I see "strace" counting 200+ file-related syscalls per request with recent linux/debian/apache/php-fpm. > > It's not so hard to get it better with other langages : when they are compiled, or when framework loading happens at server start and to at each request. > > Web projects have more and more a "compilation" phase for minimising JS, for proprocessing scss to css, it is possible to hook it some PHP processing too. > > In my university, I started using a purpose-written PHP Preprocessor to minimize runtime dependency loading. It only understands #include like in C. From a src/mywebservice/v1/some-endpoint.php it will generate and build/mywebservice/v1/some-endpoint.php with all #include "some/path/somedep.php" replaced by the file's contents. > > So the generated some-endpoint.php have no run-time dependency at all: no autoload, no include(), no require(). I think it evens maximize the gains with APC caching. > > The preprocessor also generate things like #line 2 "some/path/somedep.php" where an #include was encountered, then a thing like #line 47 "src/mywebservice/v1/some-endpoint.php" right after the end of the inclusion. In C, a great concrete example of #line importance is working with a flex/bison parser generator. > > If PHP parser interpret #line as in C, __FILE__ and __LINE__ Magic Constants will be changed to source file and line, instead of generated file and line. It could greatly improve development write-then-rerun cycle. (missing ";" at line NN , other PHP Errors, Exception details/traces) > > I hope it could unlock many use cases where "big" PHP frameworks get really hard time to try to compete with other languages equivalents. > > Do you think it's an idea that is suitable to discuss, improve and submit as an RFC ? > > Regards, > > -- > > Ludovic Pouzenc > www.pouzenc.fr > > -- > PHP Internals - PHP Runtime Development Mailing List > To unsubscribe, visit: https://www.php.net/unsub.php >
Hi Ludovic, I can’t comment much on the actual change in the parser you’re asking about, but it sounds to me like preloading (https://www.php.net/manual/en/opcache.preloading.php <https://www.php.net/manual/en/opcache.preloading.php>, introduced via https://wiki.php.net/rfc/preload <https://wiki.php.net/rfc/preload>) might be a workable solution that’s already available? Cheers Stephen

Larry Garfield

3 years ago
On Sun, Oct 23, 2022, at 8:43 AM, Ludovic Pouzenc wrote:
> Hi, > > I'm a newcomer here. I came from https://wiki.php.net/rfc/howto. > > *TL;DR:* I want to know if a proposal of a #line parsing to alter > __FILE__ and __LINE__ in PHP like it exists in C could be a good idea. > > I've searched about __LINE__ and __FILE__ in mailing archives and wiki, > I don't really find things matching. I hope it's not because "__" are > stripped before search. "Magic constants" didn't help neither. Let me > know if I am rethinking and old thing already imagined here in the past. > > *Background / use case:* I think at first about webservices, maybe the > useful also with browser/human-facing use cases. > > With PHP, it's hard to get high responsiveness under load if user code > start by loading a composer's autoload.php, then (lazy) loads a whole > framework with 10+ dependencies, then execute 1 SQL request, 1 > json_encode() then output and exit. Even with "Slim" framework that > should be minimal, using only the route feature, I see "strace" counting > 200+ file-related syscalls per request with recent > linux/debian/apache/php-fpm. > > It's not so hard to get it better with other langages : when they are > compiled, or when framework loading happens at server start and to at > each request. > > Web projects have more and more a "compilation" phase for minimising JS, > for proprocessing scss to css, it is possible to hook it some PHP > processing too. > > In my university, I started using a purpose-written PHP Preprocessor to > minimize runtime dependency loading. It only understands #include like > in C. From a src/mywebservice/v1/some-endpoint.php it will generate and > build/mywebservice/v1/some-endpoint.php with all #include > "some/path/somedep.php" replaced by the file's contents. > > So the generated some-endpoint.php have no run-time dependency at all: > no autoload, no include(), no require(). I think it evens maximize the > gains with APC caching. > > The preprocessor also generate things like #line 2 > "some/path/somedep.php" where an #include was encountered, then a thing > like #line 47 "src/mywebservice/v1/some-endpoint.php" right after the > end of the inclusion. In C, a great concrete example of #line importance > is working with a flex/bison parser generator. > > If PHP parser interpret #line as in C, __FILE__ and __LINE__ Magic > Constants will be changed to source file and line, instead of generated > file and line. It could greatly improve development write-then-rerun > cycle. (missing ";" at line NN , other PHP Errors, Exception details/traces) > > I hope it could unlock many use cases where "big" PHP frameworks get > really hard time to try to compete with other languages equivalents. > > Do you think it's an idea that is suitable to discuss, improve and > submit as an RFC ? > > Regards,
It sounds like there are numerous existing approaches to address the situation you describe that are already in use and far more effective than making constants stealthily dynamic. 1. Opcache preloading, introduced in PHP 7.4. 2. Disable the stat calls on opcache, so it doesn't even check the disk at all once something is in cache. 3. Use a persistent-process tool like React PHP, AmPHP, Revolt, and the like, so it functions more like Node.js instead of using PHP-FPM. I think you're also greatly over-estimating how large the startup cost is in practice. It's real, certainly, but unless you have several nested microservices a well-made framework should have a fairly small overhead. --Larry Garfield

Ludovic Pouzenc

3 years ago
Le 23/10/2022 à 18:02, Larry Garfield a écrit :
> On Sun, Oct 23, 2022, at 8:43 AM, Ludovic Pouzenc wrote: >> Hi, >> >> I'm a newcomer here. I came from https://wiki.php.net/rfc/howto. >> >> *TL;DR:* I want to know if a proposal of a #line parsing to alter >> __FILE__ and __LINE__ in PHP like it exists in C could be a good idea. >> >> I've searched about __LINE__ and __FILE__ in mailing archives and wiki, >> I don't really find things matching. I hope it's not because "__" are >> stripped before search. "Magic constants" didn't help neither. Let me >> know if I am rethinking and old thing already imagined here in the past. >> >> *Background / use case:* I think at first about webservices, maybe the >> useful also with browser/human-facing use cases. >> >> With PHP, it's hard to get high responsiveness under load if user code >> start by loading a composer's autoload.php, then (lazy) loads a whole >> framework with 10+ dependencies, then execute 1 SQL request, 1 >> json_encode() then output and exit. Even with "Slim" framework that >> should be minimal, using only the route feature, I see "strace" counting >> 200+ file-related syscalls per request with recent >> linux/debian/apache/php-fpm. >> >> It's not so hard to get it better with other langages : when they are >> compiled, or when framework loading happens at server start and to at >> each request. >> >> Web projects have more and more a "compilation" phase for minimising JS, >> for proprocessing scss to css, it is possible to hook it some PHP >> processing too. >> >> In my university, I started using a purpose-written PHP Preprocessor to >> minimize runtime dependency loading. It only understands #include like >> in C. From a src/mywebservice/v1/some-endpoint.php it will generate and >> build/mywebservice/v1/some-endpoint.php with all #include >> "some/path/somedep.php" replaced by the file's contents. >> >> So the generated some-endpoint.php have no run-time dependency at all: >> no autoload, no include(), no require(). I think it evens maximize the >> gains with APC caching. >> >> The preprocessor also generate things like #line 2 >> "some/path/somedep.php" where an #include was encountered, then a thing >> like #line 47 "src/mywebservice/v1/some-endpoint.php" right after the >> end of the inclusion. In C, a great concrete example of #line importance >> is working with a flex/bison parser generator. >> >> If PHP parser interpret #line as in C, __FILE__ and __LINE__ Magic >> Constants will be changed to source file and line, instead of generated >> file and line. It could greatly improve development write-then-rerun >> cycle. (missing ";" at line NN , other PHP Errors, Exception details/traces) >> >> I hope it could unlock many use cases where "big" PHP frameworks get >> really hard time to try to compete with other languages equivalents. >> >> Do you think it's an idea that is suitable to discuss, improve and >> submit as an RFC ? >> >> Regards, > It sounds like there are numerous existing approaches to address the situation you describe that are already in use and far more effective than making constants stealthily dynamic. > > 1. Opcache preloading, introduced in PHP 7.4. > 2. Disable the stat calls on opcache, so it doesn't even check the disk at all once something is in cache. > 3. Use a persistent-process tool like React PHP, AmPHP, Revolt, and the like, so it functions more like Node.js instead of using PHP-FPM. > > I think you're also greatly over-estimating how large the startup cost is in practice. It's real, certainly, but unless you have several nested microservices a well-made framework should have a fairly small overhead. > > --Larry Garfield >
Thanks for replies, I will test tools and features mentionned above.
-- Ludovic Pouzenc www.pouzenc.fr

Unnamed Person

3 years ago
> I think you're also greatly over-estimating how large the startup cost is in practice. It's real, certainly, but unless you have several nested microservices a well-made framework should have a fairly small overhead.
I want to share my experience that some users have very slow file I/O, and I regularly see `file_exists` and `stat` show up in slow code paths. This is one reason I would like to remove the warning on a failed `include`, because then instead of `if (file_exists($file)) require $file;` or similar, you can just `include $file;` it. When I brought it up last time, some people actually liked that it has warnings so... I didn't move forward. Maybe there's room for `try_include`: [$included, $retval) = try_include($file); if ($included) { // $retval will have the return value of the file } else { // $retval will have a warning/error message you can work with } I like this quite a bit less than changing `include` to not emit a warning, but it's not without merits: 1. `include` still emits a warning, so nobody relying on that will be surprised. 2. User gets to control error handling. 3. Can be polyfilled, it just costs a bit of performance. Anyway, it's a bit off topic, so I'll stop here.