PHP True Async RFC - Stage 2

On 16/03/2025 09:24, Edmond Dantes wrote:

Looks tremendous, at a glance. Thanks for your work on this. Just one quick question for now; why is `suspend()` a function and not a statement? Cheers, Bilge

On 16/03/2025 09:24, Edmond Dantes wrote:

Thank you for taking such a positive approach to the feedback, and continuing to really work hard on this! I found this version of the RFC much more digestible than the last, and have some specific comments on various sections... ## Possible Syntax

concurrency. This syntax is NOT a mandatory part of this RFC and may be adopted separately. As I said in the previous thread, I disagree with this - control flow should use keywords, not functions. We don't expose a function Loop\while(callable $condition, callable $action): void Having both just leads to confusing caveats, like:

parameters. This limitation can be overcome using the spawn operator. Specifically, I suggest keywords (and no function equivalents) for the following (names subject to bikeshedding): - spawn - spawn in $scope  - await - suspend - defer Some things proposed as methods can be easily composed from these keywords by the user: // instead of $scope->awaitDirectChildren(); Async\await_all( $scope->getDirectChildren() ); // instead of $crucialCoroutine = $boundedScope->spawnAndBound( some_function() ); $crucialCoroutine = spawn in $boundedScope some_function(); $boundedScope->boundedBy($crucialCoroutine);

has two forms ... executing a closure This leads to an ambiguity / source of confusion: function x(): int { return 42; } spawn x(); // spawns x(); spawn function(): int { return 42; } // immediately evaluates the function() expression to a Closure, and then spawns it $var = 42; spawn $var; // presumably throws an error $callable = function(): int { return 42; }; spawn $callable; // does this spawn the callable, or throw an error? function foo(): callable { return bar(...); } spawn foo(); // does this spawn foo(), or run foo() immediately and then spawn bar() ?? I suggest we instead allow "spawn" to be followed by a block, which implicitly creates and spawns a zero-argument Closure: function x(): int { return 42; } spawn x(); // spawns x(); $callable = function() { foo(); bar(); }; spawn $callable(); // spawns the Closure spawn ( function() { foo(); bar(); } )(); // legal, but unlikely in practice; note the last (), to call the closure, not just define it spawn { foo(); bar(); } // shorthand for the previous example I'm not sure about variable capture - probably we would need a use() clause for consistency with function{}, and to avoid inventing a new capture algorithm: $x = 42; spawn use ($x) { do_something($x); do_something_else($x); } ## Lifetime Limitation The RFC currently mentions "the Actor model" but doesn't actually explain what it is, or what "other languages" implement it. I imagine you wanted to avoid the explanation getting any longer, but maybe some links could be added to help people find examples of it? ## BoundedScope This API needs a bit more thought I think: * The name of "withTimeout" sounds like it will return a modified clone, and the description says it "creates" something, but the example shows it mutating an existing object * What happens if boundedBy and/or spawnAndBound are called multiple times on the same BoundedScope? Is the lifetime the shortest of those provided? Or the longest? Or just the last one called? * boundedBy should not accept "mixed", but a specific union (e.g. CancellationToken|Future|Coroutine) or interface (e.g. ScopeConstraint) ## Coroutine Scope Slots The Async\Key class doesn't belong in the Async namespace or this RFC, but as a top-level feature of the language, in its own RFC. In general, the "scope slots" API still feels over-engineered, and lacking a separation of concerns: - I'm not convinced that supporting object keys is necessary; is it something userland libraries are widely implementing, or just personal taste? - I don't understand the difference between find(), get(), findLocal(), and getLocal() - Scope and Context seem like separate concerns, which should be composed not inherited - automatic dereferencing of WeakReference seems an unnecessary piece of magic - unless I'm missing something, it just saves the user typing "->get()" I haven't worked through all the use cases, but I think the key primitives are: - Scope->getParent(): ?Scope - Scope->getContext(): Context - Coroutine->getContext(): Context - Context->set(string $key, mixed $value): void - Context->get(string $key): mixed That allows for: currentScope()->getContext()->get('scope_value'); currentScope()->getParent()->getContext()->get('inherited_value'); currentCoroutine()->getContext()->get('local_value'); A framework can build more complex inheritance relationships on top of that, e.g. function get_from_request_scope(string $key): mixed {     $scope = currentScope();     while ( $scope !== null && ! $scope->getContext()->get('acme_framework::is_request_scope') ) {         $scope = $scope->getParent();     }     return $scope?->get($key); } ## Error Handling

|await $scope|, all responsibility points receive the exception. Is the exception cloned into each of these coroutines, or are they all given exactly the same instance? An exception being thrown in multiple places "simultaneously" is hard to visualise, and I wonder if it will lead to confusing situations.

cancelled')); This looks odd - why is the caller creating the exception? Can any exception be used there? I suggest: $coroutine->cancel('some message'); // constructs an Async\CancellationException and throws it inside the coroutine

exception, as it may cause application malfunctions. This and other special behaviours suggest that this should inherit from Error rather than Exception, or possibly directly from Throwable That's all for now. To reiterate: thank you so much for working on this, and I really like the shape it's beginning to take :)

On Sun, Mar 16, 2025, at 4:24 AM, Edmond Dantes wrote:

I'm unclear here. It doesn't expose Fibers at all, or it's not even touching the C code for fibers internally? Like, would this render the existing Fiber code entirely vestigial, not just its API?

Considering that lifetime management is one of the hardest things in Rust to learn, that's not a ringing endorsement.

I'll be honest... intuitive is not the term I'd use. In fact, I didn't make it all the way through the RFC before I got extremely confused about how it all worked. First off, it desperately needs an "executive summary" section up at the top. There's a *lot* going on, and having a big-picture overview would help a ton. (For examples, see property hooks[1] and pattern matching[2].) Second, please include realistic examples. Nearly all of the examples are contrived, which doesn't help me see how I would actually use async routines or what the common patterns would be, and I therefore cannot evaluate how well the proposal treats those common cases. The first non-foobar example includes a comment "of course you should never do it like this", which makes the example rather useless. And the second is built around a code model that I would never, ever accept into a code base, so it's again unhelpful. Most of the RFC also uses examples that... have no return values. So from reading the first half of it, I honestly couldn't tell you how return values work, or if they're wrapped in a Future or something. Third, regarding syntax, I largely agree with Tim that keywords are better than functions. This is very low-level functionality, so we can and should build dedicated syntax to make it as robust and self-evident (and IDE friendly) as possible. That said, even allowing for the async or await or spawn keywords, I got super confused when the Scope object was introduced. So would the functions/keywords be shortcuts for some of the common functionality of a Scope object? If not, what's the actual difference? I got lost at that point. The first few sections of the RFC seem to read as "this RFC doesn't actually work at all, until some future RFC handles this other part." Which... no, that's not how this works. :-) As someone that has not built an async framework before (which is 99.9% of PHP developers, including those on this list), I do not see the point of half the functionality here. Especially the BoundedScope. I see no reason for it to be separate from just any other Scope. What is the difference between scope and context? I have no clue. My biggest issue, though, is that I honestly can't tell what the mental model is supposed to be. The RFC goes into detail about three different async models. Are those standard terms you're borrowing from elsewhere, or your own creation? If the former, please include citations. I cannot really tell which one the "playpen" model would fit into. I... think bottom up, but I'm not sure. Moreover, I then cannot tell which of those models is in use in the RFC. There's a passing reference to it being bottom up, I think, but it certainly looks like the No Limit model. There's a section called structured concurrency, but what it describes doesn't look a thing like the playpen-definition of structured concurrency, which as noted is my preference. It's not clear why the various positives and negatives are there; it's just presented as though self-evident. Why does bottom up lead to high memory usage, for instance? That's not clear to me. So really... I have no idea how to think about any of it. Sorry, I'm just totally lost at this point. As an aside: I used "spawn" as a throw-away keyword to avoid using "await" in a previous example. It's probably not the right word to use in most of these cases. I know some have expressed the sentiment that tightly structured concurrency is just us not trusting developers and babysitting them. To which I say... YES! The overwhelming majority of PHP developers have no experience writing async code. Their odds of getting it wrong and doing something inadvertently stupid by accident through not understanding some nuance are high. And I include myself in that. MY chances of inadvertently doing something stupid by accident are high. I *want* a design that doesn't let me shoot myself in the foot, or at least makes it difficult to do. If that means I cannot do everything I want to... GOOD! Humans are not to be trusted with manually coordinating parallelism. We're just not very good at it, as a species. Broadly speaking, I can think of three usage patterns for async in PHP (speaking, again, as someone who doesn't have a lot of async experience, so I may be missing some): 1. Fan-out. This is the "fetch all these URLs at once" type use case, which in most cases could be wrapped up into a para_map() function. (Which is exactly what Rust does.) 2. Request handlers, for persistent-process servers. Would also apply for a queue worker. 3. Throw it over the wall. This would be the logging example, or sending an email on some trigger, etc. Importantly, these are cases where there is no result needed from the sub-routine. I feel like those three seem to capture most reasonable use cases, give or take some details. (And, of course, many apps will include all three in various places.) So any proposal should include copious examples of how those three cases would look, and why they're sufficiently ergonomic. A playpen model can handle both 1 and 2. In fan out, you want the "Wait all" logic, but then you also need to think about a Future object or similar. In a request handler, you're spawning an arbitrary number of coroutines that will terminate, and you probably don't care if they have a return value. It's the "throw over the wall" cases where a playpen takes more work. As I showed previously, it can be done. It just takes a bit more setup. But if that is too much for folks, I offer a compromise position. Again, just spitballing the syntax specifics: // Creates an async scope, in which you can create coroutines. async { // Creates a new coroutine that MAY last beyond the scope of this block. // However, it MUST be a void-return function, indicating that it's going to // do work that is not relevant to the rest of this block. spawn func_call(1, 2, 3); // Creates a new coroutine that will block at the end of this async block. // The return value is a future for whatever other_function() will return. // $future may be used as though it were the type returned, but trying // to read it will block until the function completes. It may also have other // methods on it, not sure. $future = start other_function(4, 5, 6); // Queues a coroutine to get called after all "start"ed coroutines have completed // and this block is about to end. Its return value is discarded. Perhaps it should be // restricted to void-return, not sure. In this case it doesn't hurt anything. defer cleanup(7, 8, 9); // Do nothing except allow other coroutines to switch in here if they want. suspend; // Enqueues this coroutine to run in 100 ms, or slightly thereafter whenever the scheduler gets to it. timeout 100ms something(4, 5, 6); } // There is an implicit wait-all here for anything start-ed, but not for spawn-ed. I honestly cannot see a use case at this point for starting coroutines in arbitrary scopes. Only "current scope" and "global scope, let it escape." That maps to "start" and "spawn" above. If internally "spawn" gets translated to "start in the implicit async block that is the entire application", so that those coroutines will still block the whole script from terminating, that is not a detail most devs will care about. (Which also means in the global async scope, spawn and start are basically synonymous.) I can see the need for cancellation, which means probably we do need a scope object to represent the current async block. However, that's just a cancel() method, which would propagate to any child. Scheduling it can be handled by the timeout command. At this point, I do not see the use case for anything more advanced than the above (except for channels, which as I argued before could make spawn unnecessary). There may be a good reason for it, but I don't know what it is and the RFC does not make a compelling argument for why anything more is needed. I could see an argument that async $scope { ... } lets you call all of the above keywords as methods on $scope, and the keywords are essentially a shorthand for "this method on the current scope". But you could also pass the scope object around to places if you want to do dangerous things. I'm not sure if I like that, honestly, but it seems like an option. Elsewhere in the thread, Tim noted that we should unify the function call vs closure question. I used straight function calls above for simplicity, but standardizing on a closure also makes sense. Related, I've been talking with Arnaud about trying to put Partial Function Application forward again[3], assuming pipes[4] pass. If we follow the previous model, then it would implicitly provide a way to turn any function call into a delayed function call: function foo(int $a, int $b) { ... } foo(4, 5); // Calls foo() right now foo(4, 5, ...); // Creates a 0-argument closure that will call foo(4, 5) when invoked. Basically the latter is equivalent to: fn() => foo(4, 5); A 0-argument closure (because all arguments are already captured) goes by the delightful name "thunk" (as in the past tense of think, if you don't know English very well.) That likely wouldn't be ideal, but it would make standardizing start/spawn on "thou shalt provide a closure" fairly straightforward, as any function could trivially be wrapped into one. That's not necessarily the best way, but I mention it to show that there are options available if we allow related features to support each other synergistically, which I always encourage. Like Tim, I applaud you're commitment to this topic and willingness to work with feedback. But the RFC text is still a long way from a model that I can wrap my head around, much less support. [1] https://wiki.php.net/rfc/property-hooks [2] https://wiki.php.net/rfc/pattern-matching [3] https://wiki.php.net/rfc/partial_function_application [4] https://wiki.php.net/rfc/pipe-operator-v3 --Larry Garfield

On Sun, Mar 16, 2025, at 10:24, Edmond Dantes wrote:

Hey Edmond, Here are my notes:

Generally, RFCs are for changes in the language itself, not for API contracts in C. That can generally be handled in PRs, if I understand correctly.

If it can throw, then it does return values? I can foresee people abusing this for flow control and passing out (serialized) values of suspended coroutines. Especially if it is broadcast to all other coroutines awaiting it. It is probably simpler to simply allow passing a value out via suspend.

Does this mean it is an expression? So you can basically do: return suspend(); $x = [suspend(), suspend(), suspend()]; foreach ($x as $_) {} or other weird shenanigans? I think it would be better as a statement.

What happens if it throws? Why does it return NULL; why not `void` or the result of the awaited spawn?

Wouldn't it be better to throw an exception instead of silently failing? From this section, I really don't like the dual-syntax of `spawn`, it is function-like, except not. In other words, this won't behave like you would expect it to: spawn ($is_callable ? $callable : $default_callable)($value); I'm not sure what will actually happen here. --- When comparing the three different models, it would be ideal to keep to the same example for all three and describe how their execution differs between the example. Having to parse through the examples of each description is a pain.

Hmm. Do you mean this literally? So if I call a random function via spawn, it will have access to my current scope? function foo() { $x = 'bar'; } $x = 'baz'; $scope->spawn(foo(...)); echo $x; // baz or bar?? That seems like a massive footgun. I think you mean to say that it would behave like normal. If you spawn a function, it behaves like a function, if you spawn a closure, it closes over variables just like normal. Though I think it is worth defining "when" it closes over the variables -- when it executes the closure, or when it hits the spawn. Does "spawn" not provide a \Scope? --- I still don't understand the need for a special context thing. One of the most subtle footguns with go contexts is to propagate the context when it shouldn't be propagated. For example, if you are sending a value to a queue, you probably don't want to send the request context. If you did and the request was cancelled (or even just completed!), it would also cancel putting the value on the queue -- which is almost certainly what you do *not* want. Since the context is handled for you, you also have to worry about a context disappearing while you are using it, from the looks of things. This can be easily built in userland, so I'm not sure why we are defining it as part of the language.

heh, reminds me of records. --- I know I have been critical in this email, but I actually like it; for the most part. I think there are still some rough edges to sand down and polish, but it is on the right track! — Rob

Hello everyone, I’d like to ask for your help regarding the syntax. **Goal:** I want to get rid of the `BoundedScope` object while still providing a convenient built-in tool for controlling wait time. To achieve this, we could extend the `await` expression so that it allows explicitly specifying a limit. For example: ```php await $some with 5s; ``` Or a more general approach: ``` [<resultExp> = ] await <AwaitExp> [bounded <CancellationExp>]; ``` I’m concerned that no other programming language has a similar construct. On the other hand, if `Cancellation` is defined in a different way (not through syntax), it requires a separate function. For example: ```php await all([$task1, $task2], $cancellation); ``` This approach is actually used in all other languages. My question is: Should cancellation have its own syntax, or should we follow the conventional approach?

Good day, everyone. As I write more code examples, I'm starting to get annoyed by the verbosity of the `spawn in $scope` construct—especially in situations where all spawns need to happen within the same context. At the same time, in 80% of cases, it turns out that explicitly defining `$scope` is the only correct approach to avoid shooting yourself in the foot. So, it turns out that the `spawn in $scope` construct is used far more frequently than a plain `spawn`. I remembered an example that Larry came up with and decided to use it as syntactic sugar. There’s some doubt because the actual gain in characters is minimal. This block doesn't change the logic in any way. The convenience lies in the fact that within the block, it’s clear which `$scope` is currently active. However, this is more about visual organization than logical structure. Here's what I ended up with: ### Async blocks Consider the following code: ```php function generateReport(): void { $scope = Scope::inherit(); try { [$employees, $salaries, $workHours] = await Async\all([ spawn in $scope fetchEmployees(), spawn in $scope fetchSalaries(), spawn in $scope fetchWorkHours() ]); foreach ($employees as $id => $employee) { $salary = $salaries[$id] ?? 'N/A'; $hours = $workHours[$id] ?? 'N/A'; echo "{$employee['name']}: salary = $salary, hours = $hours\n"; } } catch (Exception $e) { echo "Failed to generate report: ", $e->getMessage(), "\n"; } } ``` with async ```php function generateReport(): void { try { $scope = Scope::inherit(); async $scope { [$employees, $salaries, $workHours] = await Async\all([ spawn fetchEmployees(), spawn fetchSalaries(), spawn fetchWorkHours() ]); foreach ($employees as $id => $employee) { $salary = $salaries[$id] ?? 'N/A'; $hours = $workHours[$id] ?? 'N/A'; echo "{$employee['name']}: salary = $salary, hours = $hours\n"; } } } catch (Exception $e) { echo "Failed to generate report: ", $e->getMessage(), "\n"; } } ``` #### async syntax ```php async <scope> { <codeBlock> } ```

Hello everyone, It's a nice Sunday evening, and I'd like to share some updates and thoughts from this week — kind of like a digest :) 1. Big thanks to Rowan Tommins for the syntax suggestions, ideas, and feedback. I decided to try using the `spawn block` syntax, and in practice, it turned out to be quite convenient. So I'll include it in the next phase. You can check out the details via the link: https://github.com/EdmondDantes/php-true-async-rfc/blob/main/basic.md#spawn-closure-syntax ```php function startServer(): void { async $serverSupervisor { // Secondary coroutine that listens for a shutdown signal spawn use($serverSupervisor) { await Async\signal(SIGINT); $serverSupervisor->cancel(new CancellationException("Server shutdown")); } // Main coroutine that listens for incoming connections await spawn { while ($socket = stream_socket_accept($serverSocket, 0)) { connectionHandler($socket); } }; } } ``` 2. suspend has become a statement. 3. The scope retrieval functions like currentScope, globalScope, and rootScope have been removed. This has consequences. One of them: it's no longer possible to create a "detached" coroutine. But that’s good news. 4. I decided to borrow Larry's example and create a special code block "*async block"* that interacts with coroutine Scope in a special way. ```php function startServer(): void { async $serverSupervisor { // Secondary coroutine that listens for a shutdown signal spawn use($serverSupervisor) { await Async\signal(SIGINT); $serverSupervisor->cancel(new CancellationException("Server shutdown")); } // Main coroutine that listens for incoming connections await spawn { while ($socket = stream_socket_accept($serverSocket, 0)) { connectionHandler($socket); } }; } } ``` It looks nice, even though it's syntactic sugar for: ```php $scope = new Scope(); try { await spawn in $scope { echo "Task 1\n"; }; } finally { $scope->dispose(); } ``` This syntax creates a code block that limits the lifetime of coroutines until the block is exited. It doesn't *wait*, it *limits*. Besides the fact that the block looks compact, it can be checked by static analysis for the presence of `await`, verify what exactly is being awaited, and report potential errors. In other words, such code is much easier to analyze and to establish relationships between groups of coroutines. The downside is that it's not suitable for classes with destructors. But that's not really a drawback, since there's a different approach for handling classes. 5. I decided to abandon `await all + scope`. Reason: it's too tempting to shoot yourself in the foot. Instead of `await $scope`, I want the programmer to explicitly choose what exactly they intend to wait for: only direct children or all others. If you're going to shoot yourself in the foot — do it with full awareness :) Drawback: it complicates the logic. But on the other hand, this approach makes the code better. The code only awaits the coroutines that were created inside the foreach: ```php function processAllUsers(string ...$users): array { $scope = new Scope(); foreach ($users as $user) { spawn in $scope processUser($user); } return await $scope->tasks(); } ``` Code that waits until all child coroutines — at any depth — of the launched background tasks have completed. ```php function processBackgroundJobs(string ...$jobs): array { $scope = new Scope(); foreach ($jobs as $job) { spawn with $scope processJob($users); } await $scope->all(); } ``` It doesn’t look terrible, but I’m concerned that this kind of functionality might feel “complex” from a learning curve perspective. On the other hand, Python’s approach to similar things is even more complex, largely because the language added async features in several stages. My main doubts revolve around the fact that Scope is passed implicitly between function calls. This creates multiple usage scenarios — i.e., a kind of flexibility that no other language really has. And as we know, flexibility has a dark side: it opens up ways to break everything. On one hand, this RFC effectively allows writing in the style of Go, Kotlin, C#, or even some other paradigms. On the other hand, identifying the “dark side of the force” becomes harder. If you don’t use Scope — you’re writing Go-style code. If you use Scope + all + async — it’s Kotlin. If you use Scope + tasks() — it’s more like Elixir. And you can also just pass $scope explicitly from function to function — then you get super-explicit structured concurrency. So I keep asking myself: wouldn’t it have been simpler to just implement the Go model? :) How can you know in advance that the chosen solution won’t lead to twisted practices or eventually result in anti-patterns? How can you tell if the chosen toolkit is strict enough — and not *too* flexible? These questions are the main reason why the next revision won’t be released very soon. More code and examples are needed to understand how reliable this really is. But in the meantime, you can keep an eye on this: https://github.com/EdmondDantes/php-true-async-rfc/blob/main/basic.md

Good day, everyone. Just a ping email — I haven’t disappeared, development is ongoing. Since the task has a high level of interdependency, I have to cautiously try different combinations. The second-to-last version, in trying to satisfy all requirements, turned out too complex to be taken seriously. However, the previous version was brought to a logical conclusion, which gave a clearer overall picture. Mind maps and feature-to-requirement diagrams were very helpful, as they allowed me to view the RFC as a balance between RFC complexity and code complexity. I’ve finally managed to find a good balance, though this RFC still surpasses all existing models in terms of tooling. I know minimalists won’t like it, but this solution definitely deserves consideration. Have a good day.