Everything
php
Learn Community Releases The PHP Project
Home » Project » Newsgroups » php.internals » First-class callable partial application »

First-class callable partial application

php.internals

[ ]

3 years ago
Hello internals! I'm just wondering why First class callable syntax doesn't allow partial application? Recently I stumbled across following scenario where it could be really useful: ``` public function __invoke(SendOtpCommand $command) { $this->cache->get($command->getPhone(), $this->storeOtp($command, ...)); } private function storeOtp(SendOtpCommand $command, ItemInterface $item) { // some logic } ``` In this example, I supposed that the closure created will accept a single parameter $item, and when it is called back, method storeOtp will accept both $command and $item. As it turned out to be, it doesn't really work this way. Another simplified example: ``` // partial application foo(bar(1, ...)); function foo(Closure $closure) { $closure(2); } function bar(int $a, int $b) { var_dump($a, $b); // 1, 2 } ``` Closure in foo accepts only one parameter. But when it is actually dispatched, bar is called with two arguments. Are there any pitfalls, which prevent implementation of this nifty feature?

Larry Garfield

3 years ago
On Fri, Mar 10, 2023, at 1:04 PM, Eugene Sidelnyk wrote:
> Hello internals! > I'm just wondering why First class callable syntax doesn't allow partial > application? > Recently I stumbled across following scenario where it could be really > useful: > > ``` > public function __invoke(SendOtpCommand $command) > { > $this->cache->get($command->getPhone(), $this->storeOtp($command, > ...)); > } > > private function storeOtp(SendOtpCommand $command, ItemInterface $item) > { > // some logic > } > ``` > > In this example, I supposed that the closure created will accept a single > parameter $item, and when it is called back, method storeOtp will accept > both $command and $item. As it turned out to be, it doesn't really work > this way. > > Another simplified example: > > ``` > // partial application > foo(bar(1, ...)); > > function foo(Closure $closure) > { > $closure(2); > } > > function bar(int $a, int $b) > { > var_dump($a, $b); // 1, 2 > } > ``` > > Closure in foo accepts only one parameter. But when it is actually > dispatched, bar is called with two arguments. > > Are there any pitfalls, which prevent implementation of this nifty feature?
There actually was an RFC for full partial application a few years ago: https://wiki.php.net/rfc/partial_function_application The main drawback is that in order to work, it had to do some very tricksy things with very critical parts of the code (how functions get called). Enough people felt that was too risky for the functionality and it didn't pass. First-class-callables were deliberately submitted (https://wiki.php.net/rfc/first_class_callable_syntax) as a sort of "junior version" of partial function application that could be done with far less invasive changes. I'd *love* to see some form of PFA make a come-back, either with a simpler implementation or a reduced scope that allows for a simpler implementation. I am not aware of anyone actively working on it at the moment, though. (If anyone is, please speak up!) --Larry Garfield

[ ]

3 years ago
I glanced over this RFC and suggested syntax has question marks. Each one stands for a single parameter. Since there are named arguments already, it seems to be redundant. We'd rather pass couple of named arguments and place the ... construction at the ending than twirl around with question marks in order to describe which parameters final closure will have. When we do PFA, what we think of is what we pass, not what would be necessary to pass afterwards. And yes, any feature should start from the smallest possible form. If we try to implement all at once, we would get nowhere. On Fri, Mar 10, 2023, 9:53 PM Larry Garfield <larry@garfieldtech.com> wrote:

Larry Garfield

3 years ago
On Fri, Mar 10, 2023, at 2:16 PM, Eugene Sidelnyk wrote:
> I glanced over this RFC and suggested syntax has question marks. Each one > stands for a single parameter. > > Since there are named arguments already, it seems to be redundant. We'd > rather pass couple of named arguments and place the ... construction at the > ending than twirl around with question marks in order to describe which > parameters final closure will have. > > When we do PFA, what we think of is what we pass, not what would be > necessary to pass afterwards. > > And yes, any feature should start from the smallest possible form. If we > try to implement all at once, we would get nowhere.
(Please don't top-post.) Named args were still very new at the time, so it wasn't clear the degree to which they should be required. I still don't think only supporting named args is a good idea, as that creates additional work for common cases. Also, that wouldn't have changed the complexity part that people objected to. It was the engine telling the difference at runtime between "foo($a, $b, $c)" and "foo($a, $b, ?)", which happens inside the C code for "call a function", that was the problem. Named args wouldn't have mattered. Sometimes features can start from small forms, but not always. Sometimes that just boxes you in and makes it even harder to make the more complete version. Case in point, `readonly` was the junior version of asymmetric visibility, but was implemented too quickly without thinking through all the implications, which then caused problems for actual asymmetric visibility's implementation. That's one of the reasons asymmetric visibility didn't pass. So in that case, the junior version prevented the full version from happening. Anyway, until someone is able to commit time to coming up with a better implementation this is all moot. :-) --Larry Garfield

Christian Schneider

3 years ago
Am 10.03.2023 um 20:04 schrieb Eugene Sidelnyk <zsidelnik@gmail.com>:
> Another simplified example: > > ``` > // partial application > foo(bar(1, ...));
While it is not as concise as your version I think the work-around with short arrow functions works ok unless your code is full of partial applications: foo(fn(...$a) => bar(1, ...$a)); Regards, - Chris

Robert Landers

3 years ago
On Fri, Mar 10, 2023 at 10:56 PM Christian Schneider <cschneid@cschneid.com> wrote:
> > Am 10.03.2023 um 20:04 schrieb Eugene Sidelnyk <zsidelnik@gmail.com>: > > Another simplified example: > > > > ``` > > // partial application > > foo(bar(1, ...)); > > While it is not as concise as your version I think the work-around with short arrow functions works ok unless your code is full of partial applications: > foo(fn(...$a) => bar(1, ...$a)); > > Regards, > - Chris > > -- > PHP Internals - PHP Runtime Development Mailing List > To unsubscribe, visit: https://www.php.net/unsub.php >
Hey Eugene I actually started on an implementation with this exact syntax this past summer. I stopped after seeing the RFC was already declined years ago. However, there are still some tricky parts to work out. Right now, the "first-class callable" gets (basically) turned into a callable string at the first possible instant, just after parsing. This has to be thrown away, or handled in some other way. In my implementation, I went with treating that as a special case, and basically considering the partial application as 'syntax sugar'. There are also other issues with the grammar, which I'll get to in a few minutes. My "syntax sugar" implementation was that this: return $this->cache->get($command->getPhone(), $this->storeOtp($command, ...)); gets turned into this: return fn(...$x) => $this-cache->get($command->getPhone(), $this->storeOtp($command, ...$x)); There were a number of edge cases to solve, but I stopped implementation before I got there. The biggest issue was that the grammar became a little bit ambiguous. I'm sure someone smarter than me can figure out how to define the grammar in a way that isn't ambiguous... but the gist is that the triple-dot is also used for array unpacking, thus I ended up disabling that code path to get it to work. I can probably clean up my code and submit a PR, if anyone would be interested. Cheers, Rob

Rowan Collins

3 years ago
Hi all, On Sat, 11 Mar 2023 at 22:48, Robert Landers <landers.robert@gmail.com> wrote:
> My "syntax sugar" implementation was that this: > > return $this->cache->get($command->getPhone(), $this->storeOtp($command, > ...)); > > gets turned into this: > > return fn(...$x) => $this-cache->get($command->getPhone(), > $this->storeOtp($command, ...$x)); > > There were a number of edge cases to solve, but I stopped > implementation before I got there. >
As someone who voted for the RFC a couple of years ago, and was disappointed it didn't pass, I'm pleased to see more interest in this. It's worth noting that the previous implementation was reached after a lot of different experiments, and there were a few requirements that guided it, including (in no particular order, and I've probably forgotten something): * Placeholders need to be possible for any parameter, not just left-to-right; e.g. it must be possible to apply the callback parameter to array_filter (which is the 2nd param), producing a callable that accepts an array to filter * It should not be possible to partially apply a non-existent function; this is one of the big advantages of first-class callable syntax over existing string-based callables: https://3v4l.org/Vr9oq#v8.2.3 * Errors from passing incorrect parameters should show as an error in the actual call, not the wrapper; compare existing FCC behaviour with a manual lambda: https://3v4l.org/Q2H2Z#v8.2.3 * The generated callable should have correct parameter type and name information visible in reflection (closely related to previous point) * The relationship with named parameters and variadics needs to at least be well-defined, even if it's "error with a possibility of expanding functionality later" (this caused a lot of pain in some of the implementation ideas) * Repeated partial application (currying and similar) should ideally not result in a whole stack of wrappers; that is $f = foo(42, ?, ?); $g = $f(69, ?); should result in the same object as $g = foo(42, 69, ?); This is definitely one of those features where "the devil is in the details", and a simpler implementation is possible, but may not be desirable. Regards,
-- Rowan Tommins [IMSoP]

Robert Landers

3 years ago
On Mon, Mar 13, 2023 at 12:28 PM Rowan Tommins <rowan.collins@gmail.com> wrote:
> > Hi all, > > On Sat, 11 Mar 2023 at 22:48, Robert Landers <landers.robert@gmail.com> > wrote: > > > My "syntax sugar" implementation was that this: > > > > return $this->cache->get($command->getPhone(), $this->storeOtp($command, > > ...)); > > > > gets turned into this: > > > > return fn(...$x) => $this-cache->get($command->getPhone(), > > $this->storeOtp($command, ...$x)); > > > > There were a number of edge cases to solve, but I stopped > > implementation before I got there. > > > > > As someone who voted for the RFC a couple of years ago, and was > disappointed it didn't pass, I'm pleased to see more interest in this. > > It's worth noting that the previous implementation was reached after a lot > of different experiments, and there were a few requirements that guided it, > including (in no particular order, and I've probably forgotten something): > > * Placeholders need to be possible for any parameter, not just > left-to-right; e.g. it must be possible to apply the callback parameter to > array_filter (which is the 2nd param), producing a callable that accepts an > array to filter > * It should not be possible to partially apply a non-existent function; > this is one of the big advantages of first-class callable syntax over > existing string-based callables: https://3v4l.org/Vr9oq#v8.2.3 > * Errors from passing incorrect parameters should show as an error in the > actual call, not the wrapper; compare existing FCC behaviour with a manual > lambda: https://3v4l.org/Q2H2Z#v8.2.3 > * The generated callable should have correct parameter type and name > information visible in reflection (closely related to previous point) > * The relationship with named parameters and variadics needs to at least be > well-defined, even if it's "error with a possibility of expanding > functionality later" (this caused a lot of pain in some of the > implementation ideas) > * Repeated partial application (currying and similar) should ideally not > result in a whole stack of wrappers; that is $f = foo(42, ?, ?); $g = > $f(69, ?); should result in the same object as $g = foo(42, 69, ?); > > This is definitely one of those features where "the devil is in the > details", and a simpler implementation is possible, but may not be > desirable. > > Regards, > -- > Rowan Tommins > [IMSoP]
Hey Rowan,
> This is definitely one of those features where "the devil is in the > details", and a simpler implementation is possible, but may not be > desirable.
My approach was more of an iterative one. 1. Get left-right done so that $x = something($left, $right, ...); would be allowed, but not $x = something($left, ..., $right); This would bring some immediate benefits, as initially proposed. I'd also propose that variadic arguments wouldn't be allowed, thus array_map(..., $x); would be allowed, but not array_map($callback, ...); 2. Implement middle/variadic arguments I felt that this could be it's own RFC once the above is implemented. As you said, the devil is in the details... what does array_map($callback, ..., $right) do? Is the partial application variadic? These are things that I felt couldn't realistically be answered until we saw some real-life usage and feedback from (1) above. 3. Optimizations This would be when we flatten the callback chain OR maybe by now, the genius(es) working on the opcache would have already beaten everyone to the punch.

Rowan Collins

3 years ago
On 13 March 2023 18:44:48 GMT, Robert Landers <landers.robert@gmail.com> wrote:
>My approach was more of an iterative one. > >1. Get left-right done so that > >$x = something($left, $right, ...); > >would be allowed, but not > >$x = something($left, ..., $right); > >This would bring some immediate benefits, as initially proposed.
There are three problems I see with that: 1) The choice of syntax for the first version immediately limits the options for later additions, so you still need to discuss them. (IIRC, the proposal that went to vote had separate ? and ... tokens, but a number of different variations were experimented with.) 2) Most of the goals I mentioned above are not about the syntax, and are no easier for left-to-right - for instance, copying the type information for use in reflection and error messages. 3) Unlike purely or primarily functional languages, PHP's standard library is not at all designed with parameter orders that lend themselves to left-to-right application. I gave array_filter as an example, because it's exactly the kind of function people will want to use PFA for, but wouldn't be able to with left-to-right only. I don't want to put you off exploring this idea, though, so feel free to take this all with as many pinches of salt as you want. Regards,
-- Rowan Tommins [IMSoP]

Larry Garfield

3 years ago
On Mon, Mar 13, 2023, at 3:36 PM, Rowan Tommins wrote:
> On 13 March 2023 18:44:48 GMT, Robert Landers <landers.robert@gmail.com> wrote: >>My approach was more of an iterative one. >> >>1. Get left-right done so that >> >>$x = something($left, $right, ...); >> >>would be allowed, but not >> >>$x = something($left, ..., $right); >> >>This would bring some immediate benefits, as initially proposed. > > > There are three problems I see with that: > > 1) The choice of syntax for the first version immediately limits the > options for later additions, so you still need to discuss them. (IIRC, > the proposal that went to vote had separate ? and ... tokens, but a > number of different variations were experimented with.) > > 2) Most of the goals I mentioned above are not about the syntax, and > are no easier for left-to-right - for instance, copying the type > information for use in reflection and error messages. > > 3) Unlike purely or primarily functional languages, PHP's standard > library is not at all designed with parameter orders that lend > themselves to left-to-right application. I gave array_filter as an > example, because it's exactly the kind of function people will want to > use PFA for, but wouldn't be able to with left-to-right only.
Personally, I consider this the easiest thing to "let go of." As has been discussed numerous times, all of the most used array functions need to be redesigned to work with iterables, and in many cases make more sense. That would be a natural time to also revisit parameter order to fit with whatever partial application syntax was in use. There are limitations to that (a left-to-right-only approach would then preclude optional arguments, which is not ideal), but "compatibility with the array functions written in 1996" is probably my least cared-about criteria. That said, full support as the original RFC had would be ideal. It's doing it in a clean and performant fashion that is the challenge. --Larry Garfield

Rowan Collins

3 years ago
On 13/03/2023 20:44, Larry Garfield wrote:
> As has been discussed numerous times, all of the most used array functions need to be redesigned to work with iterables, and in many cases make more sense. That would be a natural time to also revisit parameter order to fit with whatever partial application syntax was in use.
It's not just the array functions, though, it's every single function built into PHP, and an even longer list of userland library and framework functions; and there will always be competing reasons for preferring one signature over another. What attracts me about features like PFA is precisely that they let you work in new ways *without* having to rewrite all of that. Some more examples of placeholder-first application, from a quick skim through the documentation: $escape = htmlspecialchars(?, ENT_XML1); $containsAt = str_contains(?, '@'); $priceFormatter = number_format(?, 2, ',', '.'); $addSigToFile = file_put_contents(?, $signature, FILE_APPEND); $takeOwnership = chown(?, get_current_user()); $encode = json_encode(?, JSON_THROW_ON_ERROR | JSON_PRESERVE_ZERO_FRACTION); $unserialize = unserialize(?, ['allowed_classes' => false]); $isLogger = is_subclass_of(?, LoggerInterface::class, false); I'm sure I could look through Laravel's documentation, or Symfony's, and find examples there too. Regards,
-- Rowan Tommins [IMSoP]

Bob Weinand

3 years ago
> Am 14.03.2023 um 10:16 schrieb Rowan Tommins <rowan.collins@gmail.com>: > > On 13/03/2023 20:44, Larry Garfield wrote: >> As has been discussed numerous times, all of the most used array functions need to be redesigned to work with iterables, and in many cases make more sense. That would be a natural time to also revisit parameter order to fit with whatever partial application syntax was in use. > > > It's not just the array functions, though, it's every single function built into PHP, and an even longer list of userland library and framework functions; and there will always be competing reasons for preferring one signature over another. What attracts me about features like PFA is precisely that they let you work in new ways *without* having to rewrite all of that. > > Some more examples of placeholder-first application, from a quick skim through the documentation: > > $escape = htmlspecialchars(?, ENT_XML1); > $containsAt = str_contains(?, '@'); > $priceFormatter = number_format(?, 2, ',', '.'); > $addSigToFile = file_put_contents(?, $signature, FILE_APPEND); > $takeOwnership = chown(?, get_current_user()); > $encode = json_encode(?, JSON_THROW_ON_ERROR | JSON_PRESERVE_ZERO_FRACTION); > $unserialize = unserialize(?, ['allowed_classes' => false]); > $isLogger = is_subclass_of(?, LoggerInterface::class, false); > > I'm sure I could look through Laravel's documentation, or Symfony's, and find examples there too. > > Regards, > > -- > Rowan Tommins > [IMSoP]
Hey Rowan, do we actually need *positional* partial application, after a ... token? Would it not be enough, to simply forbid positional arguments after a ... and just allow named arguments? These already have well defined position independent semantics. There may be some desire for a single argument placeholder later on, but this can be introduced later, separately. Bob

Rowan Collins

3 years ago
On Tue, 14 Mar 2023 at 10:39, Bob Weinand <bobwei9@hotmail.com> wrote:
> Hey Rowan, > > do we actually need *positional* partial application, after a ... token? > > Would it not be enough, to simply forbid positional arguments after a ... > and just allow named arguments? These already have well defined position > independent semantics. > > There may be some desire for a single argument placeholder later on, but > this can be introduced later, separately. >
Yes, named parameters would certainly be better than left-to-right only. It's definitely less elegant, though, and given that PFA is largely short-hand for a short closure anyway, I think conciseness is quite an important aim. To take a couple of the above examples, and compare existing short closure, fully positional PFA, and named-after-placeholder PFA: $isLogger = fn($object) => is_subclass_of($object, LoggerInterface::class, false); $isLogger = is_subclass_of(?, LoggerInterface::class, false); $isLogger = is_subclass_of(..., class: LoggerInterface::class, allow_string: false); $priceFormatter = fn(float $num) => number_format($num, 2, ',', '.'); $priceFormatter = number_format(?, 2, ',', '.'); $priceFormatter = number_format(..., decimals: 2, decimal_separator: ',', thousands_separator: '.'); Arguably the named param version is more explicit, but in some cases it's significantly longer than manually defining a closure, whereas fully positional PFA is always shorter. Regards,
-- Rowan Tommins [IMSoP]

[ ]

3 years ago
On Tue, Mar 14, 2023 at 12:39 PM Bob Weinand <bobwei9@hotmail.com> wrote:
> > > Am 14.03.2023 um 10:16 schrieb Rowan Tommins <rowan.collins@gmail.com>: > > > > On 13/03/2023 20:44, Larry Garfield wrote: > >> As has been discussed numerous times, all of the most used array > functions need to be redesigned to work with iterables, and in many cases > make more sense. That would be a natural time to also revisit parameter > order to fit with whatever partial application syntax was in use. > > > > > > It's not just the array functions, though, it's every single function > built into PHP, and an even longer list of userland library and framework > functions; and there will always be competing reasons for preferring one > signature over another. What attracts me about features like PFA is > precisely that they let you work in new ways *without* having to rewrite > all of that. > > > > Some more examples of placeholder-first application, from a quick skim > through the documentation: > > > > $escape = htmlspecialchars(?, ENT_XML1); > > $containsAt = str_contains(?, '@'); > > $priceFormatter = number_format(?, 2, ',', '.'); > > $addSigToFile = file_put_contents(?, $signature, FILE_APPEND); > > $takeOwnership = chown(?, get_current_user()); > > $encode = json_encode(?, JSON_THROW_ON_ERROR | > JSON_PRESERVE_ZERO_FRACTION); > > $unserialize = unserialize(?, ['allowed_classes' => false]); > > $isLogger = is_subclass_of(?, LoggerInterface::class, false); > > > > I'm sure I could look through Laravel's documentation, or Symfony's, and > find examples there too. > > > > Regards, > > > > -- > > Rowan Tommins > > [IMSoP] > > Hey Rowan, > > do we actually need *positional* partial application, after a ... token? > > Would it not be enough, to simply forbid positional arguments after a ... > and just allow named arguments? These already have well defined position > independent semantics. > > There may be some desire for a single argument placeholder later on, but > this can be introduced later, separately. > > Bob
Personally, I agree with Robert. It seems logical to have ... only at the end of arguments list. Anything else can be bound by positional and named arguments. For instance: ``` str_contains(?, '@'); // do you remember if arg 2 is a needle or a haystack? str_contains(needle: '@', ...); // far more readable when explicitly specified explode(?, $str) // do you remember that argument 2 is actual string being exploded (not a delimiter)? explode(string: $str, ...) // delimiter will be provided to the closure ``` On the first stage, maybe it's better to have only positional arguments available without a possibility to provide named ones. I guess this will reveal a lot of pitfalls to consider. For instance, if some third-party library function signature has a signature with a variadic parameter `foo($bar, $baz, ...$x)`. We make a closure for it like this `foo(baz: $b, ...)` and it works fine. Later on authors of a library rename $baz into $tar and it gets unnoticed. Hence, the new signature is `foo($bar, $tar, ...$x)`. How would `foo(baz: $b, ...)` work? Will it put ['baz' => $b] into an $x argument? Firstly, it would make sense to only allow simple use cases of PFA. E.g. only positional parameters. Hence, it won't be possible to use PFA if we want to bind non-prefix parameters. Though, well-designed functions, which declare parameters starting from the most generic unto most specific, will perfectly fit into PFA.