[RFC] Configurable callback to dump results of expressions in `php -a`

php.internals

tyson andre

5 years ago
Hi internals, I've created a new RFC https://wiki.php.net/rfc/readline_interactive_shell_result_function adding a way to configure a callback to be called to dump the value of single-expression statements from `php -a` (and an ini setting that can be disabled to prevent that callback from being called). Many REPLs (Read-Eval-Print Loops) for other programming languages that I'm familiar with print a (possibly truncated) representation of the result of expressions, but PHP doesn't. It would be useful to allow users to extend the functionality of the default interactive php shell (php -a), possibly with auto_prepend_file or through use of command wrappers/aliases, or after loading the shell. Prior to this RFC, there was no way to extend the interactive php shell in this way. (I've seen https://github.com/bobthecow/psysh mentioned as an alternative for php -a while investigating this, but that's a shell written from scratch, and doesn't have some functionality from php -a such as tolerance of fatal errors) Because PHP's interactive shell is written in C, adding new features or bug fixes is inconvenient - it would require a lot of time getting familiar with C programming, PHP's internals and memory management, and with PHP's internal C ast representation. It would be easier and more accessible to extend PHP's interactive shell through code written in PHP rather than code written in C. This was among the proposed enhancements I brought up in https://externals.io/message/111073 . An earlier version of the implementation was also created months ago - https://github.com/php/php-src/pull/5962 Thanks, - Tyson

tyson andre

5 years ago
Hi internals, > I've created a new RFC https://wiki.php.net/rfc/readline_interactive_shell_result_function > adding a way to configure a callback to be called to dump the value of single-expression statements from `php -a` > (and an ini setting that can be disabled to prevent that callback from being called). > > Many REPLs (Read-Eval-Print Loops) for other programming languages that I'm familiar with > print a (possibly truncated) representation of the result of expressions, but PHP doesn't. > It would be useful to allow users to extend the functionality of the default interactive php shell (php -a), > possibly with auto_prepend_file or through use of command wrappers/aliases, or after loading the shell. > Prior to this RFC, there was no way to extend the interactive php shell in this way. > (I've seen https://github.com/bobthecow/psysh mentioned as an alternative for php -a while investigating this, > but that's a shell written from scratch, and doesn't have some functionality from php -a such as tolerance of fatal errors) A reminder that I plan to start the vote on https://wiki.php.net/rfc/readline_interactive_shell_result_function on 2021-01-03 Thanks, - Tyson

Nikita Popov

5 years ago
On Sun, Dec 20, 2020 at 11:36 PM tyson andre <tysonandre775@hotmail.com> wrote:
> Hi internals, > > I've created a new RFC > https://wiki.php.net/rfc/readline_interactive_shell_result_function > adding a way to configure a callback to be called to dump the value of > single-expression statements from `php -a` > (and an ini setting that can be disabled to prevent that callback from > being called). > > Many REPLs (Read-Eval-Print Loops) for other programming languages that > I'm familiar with > print a (possibly truncated) representation of the result of expressions, > but PHP doesn't. > It would be useful to allow users to extend the functionality of the > default interactive php shell (php -a), > possibly with auto_prepend_file or through use of command > wrappers/aliases, or after loading the shell. > Prior to this RFC, there was no way to extend the interactive php shell in > this way. > (I've seen https://github.com/bobthecow/psysh mentioned as an alternative > for php -a while investigating this, > but that's a shell written from scratch, and doesn't have some > functionality from php -a such as tolerance of fatal errors) > > Because PHP's interactive shell is written in C, adding new features or > bug fixes is inconvenient - it would require a lot > of time getting familiar with C programming, PHP's internals and memory > management, > and with PHP's internal C ast representation. > It would be easier and more accessible to extend PHP's interactive shell > through code written in PHP rather than code written in C. > > This was among the proposed enhancements I brought up in > https://externals.io/message/111073 . > An earlier version of the implementation was also created months ago - > https://github.com/php/php-src/pull/5962 >
Reading through the linked earlier mail, you have quite a few additional hooks in mind, which might need a significant amount of additional userland code (such as a parser implementation) to usefully implement. At that point I'm wondering what the benefit of this hybrid approach is, relatively to a userland implementation like psysh. That is: a) Assuming all the hooks have been implemented, what additional functionality is the interactive shell implementation itself actually providing? b) Is it possible to go the other way around, and expose that additional functionality to userland instead? You do mention fatal error tolerance as one distinguishing feature -- is there anything beyond that? Regards, Nikita

tyson andre

5 years ago

Rowan Collins

5 years ago
On 03/01/2021 16:11, tyson andre wrote:
> - The benefit is that dumping the result of expressions improves the default experience. > psysh wouldn't be installed by default when a new developer is learning php through the php manual, or when sshed into a remote server.
It doesn't feel to me that you've really answered Nikita's question: if all the code using these hooks is going to be distributed as userland code anyway, then they're not going to improve the default experience. It feels like we need to go in one of two directions: a) Build a full-featured official REPL with all of these improvements completely implemented out of the box. Limited extension hooks might still be desirable to build custom versions for frameworks etc, but they could be more targeted - for custom input, it could be "register meta-command"; for custom output, we already have __debugInfo() at the class level. b) Expose the magic behaviour needed for something like PsySh to do everything `php -a` already can, and leave the rest to userland. So far, the only mentioned requirement is a special form of eval() that swallows fatal errors. My feeling is that the current mood of the community favours (b) rather than (a); the most obvious example is that PHP used to bundle a PEAR executable, but Composer remains an entirely external project. Is there a reason not to aim for the same "de facto standard" for a REPL? Regards,
-- Rowan Tommins [IMSoP]

tyson andre

5 years ago
Hi Rowan Tommins, > > - The benefit is that dumping the result of expressions improves the default experience. > >    psysh wouldn't be installed by default when a new developer is learning php through the php manual, or when sshed into a remote server. > > It doesn't feel to me that you've really answered Nikita's question: if > all the code using these hooks is going to be distributed as userland > code anyway, then they're not going to improve the default experience. I was saying that I'd planned to propose defaults if this passed. __debugInfo() is a slight improvement, but the human-readable representation of an object isn't always the same thing as the debug representation of the object. A human-readable representation might be `Point(x: 1, y: 2)`, where var_dump or var_export(`__set_state`) is much longer, and I don't believe var_dump is a one size fits all solution for both simple and recursive data structures, especially since `__debugInfo` predates the repl. ``` php > class Point { public function __construct(public int $x, public int $y) {} } php > var_export(new Point(1, 2)); Point::__set_state(array( 'x' => 1, 'y' => 2, )) ``` > It feels like we need to go in one of two directions: > a) Build a full-featured official REPL with all of these improvements > completely implemented out of the box. Limited extension hooks might > still be desirable to build custom versions for frameworks etc, but they > could be more targeted - for custom input, it could be "register > meta-command"; for custom output, we already have __debugInfo() at the > class level. I'd be happy as long as we made progress on improving the interactive shell. Psysh is 2.4MB as a compiled phar release and larger if distributed with library/application releases (e.g. on remote servers). default extension hooks would likely be much smaller. > b) Expose the magic behaviour needed for something like PsySh to do > everything `php -a` already can, and leave the rest to userland. So far, > the only mentioned requirement is a special form of eval() that swallows > fatal errors. That may or may not be possible to do through creating a new `unsafe_eval` PECL (or only exposing it for interactive sessions) - I'd have to check. Something like `unsafe_eval(string $code, array &$variables): mixed result` (or throw UnsafeFatalError) The usual caveats about not using it in production would apply - the php compiler treats eval differently in that it has access to the caller's scope. https://github.com/bobthecow/psysh/blob/master/src/ExecutionLoopClosure.php uses eval(), but there's actually a lot of heuristics to avoid calling eval() on code with common known fatal errors. (it uses the readline/libedit PHP module if it's available, but doesn't load readline_cli.c) The ext/readline/readline_cli.c overrides `EG(bailout)` with zend_try macros to recover from fatal errors ```c zend_try { zend_eval_stringl(code, prompt_end - prompt_spec - 1, NULL, "php prompt code"); } zend_end_try(); ``` > My feeling is that the current mood of the community favours (b) rather > than (a); the most obvious example is that PHP used to bundle a PEAR > executable, but Composer remains an entirely external project. Is there > a reason not to aim for the same "de facto standard" for a REPL? Even if we do start distributing a better alternative shell with php or endorsing an alternative in docs, it's still possible to incrementally improve `php -a` It seems like a significant missing feature to omit printing expression results from `php -a`; people who do use `php -a` may wish to continue using that feature set (e.g. `#setting_name=value` to set/dump ini variables) but still benefit from minor improvements such as this RFC. https://github.com/bobthecow/psysh/issues/462 mentions psysh isn't a drop-in replacement. E.g. on shared hosting or when debugging on a remote host, it may be inconvenient to download psysh and faster to use `php -a` https://www.php.net/manual/en/features.commandline.interactive.php only mentions `php -a`. Perhaps it should mention external userland shells with a note that they're developed independently if the REPL rarely receives updates in favor of other functionality. - Someone learning from the php.net manual or a tutorial with minimal dependencies wouldn't install psysh right now. Aside: the php manual doesn't mention composer except in a few PECL extension docs, but people seem to figure out how to use composer out of necessity and library installation docs. Interactive shells wouldn't be as advertised Cheers, -Tyson

Rowan Collins

5 years ago
On 03/01/2021 20:45, tyson andre wrote:
> I was saying that I'd planned to propose defaults if this passed.
Then I guess my feedback can be summarised as: propose these default implementations first, rather than last. Start by improving the out-of-the-box experience, and then look at what hooks userland tools might want to customise that (which might involve something "smarter" than replacing the entire output routine).
> Psysh is 2.4MB as a compiled phar release and larger if distributed with library/application releases (e.g. on remote servers). > default extension hooks would likely be much smaller.
That was a lot of data when I used to carry floppy disks around in my school backpack. It sounds pretty reasonable for a feature-rich command-line tool in 2021.
> Someone learning from the php.net manual or a tutorial with minimal dependencies wouldn't install psysh right now.
To be honest, I doubt most users ever come across php -a either, or know that they need to install ext/readline to unlock its features (I'm pretty sure that's not installed by default on Ubuntu, for instance). I'm supportive of the idea of changing that, but I don't think the RFC as currently presented particularly helps that cause. Regards,
-- Rowan Tommins [IMSoP]

Rowan Collins

5 years ago
On 03/01/2021 20:45, tyson andre wrote:
> __debugInfo() is a slight improvement, but the human-readable representation of an object isn't always the same thing as the debug representation of the object. > A human-readable representation might be `Point(x: 1, y: 2)`, where var_dump or var_export(`__set_state`) > is much longer, and I don't believe var_dump is a one size fits all solution for both simple and recursive data structures, > especially since `__debugInfo` predates the repl.
A couple of notes on this: - The readline-based shell for "php -a" was added in PHP 5.1 [https://www.php.net/manual/en/features.commandline.interactive.php] and __debugInfo not until 5.6 [https://wiki.php.net/rfc/debug-info] - I agree that the existing debug outputs are quite verbose, but I don't think that's a problem unique to the REPL. var_export() is constrained to render valid PHP code, but print_r() and var_dump() could and perhaps should represent objects more compactly. - That compact representation of a Point would be useful pretty much everywhere anyone wanted debug output. It would also be possible to build it on top of the existing functionality (key-value pairs from __debugInfo if defined, else all properties). - Allowing objects to overload the output seems much preferable to the formatting function having to know all the special cases, so I'm not convinced of the need to hook the entire output for the shell. - I can't find any references off-hand, but I'm pretty sure popular REPLs in other languages take that approach: use existing pretty-printing mechanisms from the language, which in turn can be overloaded by individual types / classes. Regards,
-- Rowan Tommins [IMSoP]

tyson andre

5 years ago
Hi internals, > I've created a new RFC https://wiki.php.net/rfc/readline_interactive_shell_result_function > adding a way to configure a callback to be called to dump the value of single-expression statements from `php -a` > (and an ini setting that can be disabled to prevent that callback from being called). > > Many REPLs (Read-Eval-Print Loops) for other programming languages that I'm familiar with > print a (possibly truncated) representation of the result of expressions, but PHP doesn't. > It would be useful to allow users to extend the functionality of the default interactive php shell (php -a), > possibly with auto_prepend_file or through use of command wrappers/aliases, or after loading the shell. > Prior to this RFC, there was no way to extend the interactive php shell in this way. > (I've seen https://github.com/bobthecow/psysh mentioned as an alternative for php -a while investigating this, > but that's a shell written from scratch, and doesn't have some functionality from php -a such as tolerance of fatal errors) > > Because PHP's interactive shell is written in C, adding new features or bug fixes is inconvenient - it would require a lot > of time getting familiar with C programming, PHP's internals and memory management, > and with PHP's internal C ast representation. > It would be easier and more accessible to extend PHP's interactive shell through code written in PHP rather than code written in C. > > This was among the proposed enhancements I brought up in https://externals.io/message/111073 . > An earlier version of the implementation was also created months ago -  https://github.com/php/php-src/pull/5962 Some of the feedback had mentioned that adding the ability to dump the results of expressions in the default shell only made sense if there was a default implementation included with this RFC. I plan to start a straw poll shortly, to measure interest with/without a proposed result expression dumper loaded by default. https://wiki.php.net/rfc/readline_interactive_shell_result_function_straw_poll Thanks, - Tyson

tyson andre

5 years ago
Hi internals, > I've created a new RFC https://wiki.php.net/rfc/readline_interactive_shell_result_function > adding a way to configure a callback to be called to dump the value of single-expression statements from `php -a` > (and an ini setting that can be disabled to prevent that callback from being called). > > Many REPLs (Read-Eval-Print Loops) for other programming languages that I'm familiar with > print a (possibly truncated) representation of the result of expressions, but PHP doesn't. > It would be useful to allow users to extend the functionality of the default interactive php shell (php -a), > possibly with auto_prepend_file or through use of command wrappers/aliases, or after loading the shell. > Prior to this RFC, there was no way to extend the interactive php shell in this way. > (I've seen https://github.com/bobthecow/psysh mentioned as an alternative for php -a while investigating this, > but that's a shell written from scratch, and doesn't have some functionality from php -a such as tolerance of fatal errors) I've updated https://wiki.php.net/rfc/readline_interactive_shell_result_function after feedback and plan to start voting in a few days. This RFC will make the interactive shell dump non-null expression results with var_dump()/var_export() by default (https://externals.io/message/112568#112719 , https://wiki.php.net/rfc/readline_interactive_shell_result_function_straw_poll) - The dumping of expression results can be disabled through a system ini setting or at runtime ``` $ php -a Interactive shell php > 0.5*2; => 1.0 ``` Thanks, - Tyson