Proposal: Expanded iterable helper functions and aliasing iterator_to_array in `iterable\` namespace

Hi On 10/28/22 15:45, tyson andre wrote:

I wonder if this should be made an alias or if the introduction of the namespace this is a good opportunity to revisit the `$preserve_keys = true` default choice. See also this email in the voting discussion for a previous iter*able*_to_array() RFC: https://externals.io/message/102562#102616

I would not make the callable optional. If one really wants to filter on the falsyness of the items, `any($i, fn ($v) => !!$v)` is easy enough. I'd likely also swap the order of parameters, having the callable first is more natural with a possible pipe operator or partial application. It also is arguably more readable when using nested function calls, because the callback appears immediately beside the function name: (Callback first) iterable\count( iterable\filter( fn ($v) => $v > 5 iterable\map( fn ($v) => strlen($v), $iterable ) ) ) vs (Callable last) iterable\count( iterable\filter( iterable\map( $iterable, fn ($v) => strlen($v) ), fn ($v) => $v > 5 ) )

It appears you mixed up the descriptions here.

Your proposals look good to me. Especially `any` and `all` are something I'm missing somewhat regularly. Is there a reason there is no `iterable\map()` and `iterable\filter()` (that I used in my example above)? Best regards Tim Düsterhus

I support this general direction :thumbsup:

At the very least, I'd like to also see `to_array_list` which ensures that the `array_is_list` invariant is upheld by only using values from the iterable. Variants for assoc arrays have to deal with the fact that iterables have duplicate keys, and also may contain key types which don't work as array keys. 1. I think later values overwriting previous values of the same is generally agreeable. 2. However, what to do with things that aren't key types I can see having a lot of discussion. Some people probably just want them to silently filter out, some people want a warning but to proceed with the list, others may want a TypeError. I suppose we can make 3 functions, one for each case? - `to_assoc_array_silent` which drops things which don't work as array keys. - `to_assoc_array_warn` which drops things which don't work as array keys but also emits a warning. - `to_assoc_array_throw` which throws a TypeError if a key doesn't work as an array key. Not sure on these assoc things, but I'm pretty sure `to_array_list` should be included in the very first version.

:thumbsup:

:thumbsup:

I would prefer to drop the default and have an Option return type instead, but we don't have one today.

:thumbsup: If we figure out an optional/result type then I think `iterable\reduce` which just uses the first item as the initial would be very helpful, but we have to deal with the empty iterable case so I'm happy to leave it out for now since we don't have Option.

I suppose this should take an optional callback that users can provide for a custom definition of uniqueness?

Not sure on this one, but the description doesn't match the signature :) Anything which does comparison/uniqueness check should have a callback (or optional callback) for specifying how to do that.

I assume this doesn't care about key uniqueness -- it takes the keys and makes them values, in the same order they were returned? In any case, I don't think this should return an iterable, not array, to allow for lazy operations in a chain, while also allowing for using an array as an optimization.

What do they return on empty? I would prefer to delay these and try to get an Option type. Otherwise, they require doing a lot of variants like the following to be ergonomic: - first_or($iterable, mixed $default) - first_or_else($iterable, callable $callback) - last_or($iterable, mixed $default) - last_or_else($iterable, callable $callback) But if we can return an option: - first($iterable): Option - last($iterable): Option Then the other parts move to the Option API: - `iterable\first($iterable)->unwrap_or($default)` - `iterable\first($iterable)->unwrap_or_else(fn () => $whatever)` This general feedback also applies to `iterable\find()`.

I like the verbose names when there are variants, for instance `fold` and `fold_with_keys` (where we provide both key and value). I definitely do not like it when we change function signatures of callbacks based on `flags` parameters like some PHP functions do today. Phew, I'm not sure the mailing list is the best way to the tidbits of the APIs! In any case, I strongly support the direction.

On Fri, Oct 28, 2022, at 8:45 AM, tyson andre wrote:

Oh, a topic near and dear to me. :-) I'm going to try and respond to both the OP and some other responses together here. First off, I am generally in favor of improving PHP's iterable story, so consider me on board on the concept. Second, I have similar user-space utilities that were intended for pipe usage available in a library (since Levi mentioned pipe compatibility). I learned some very important things from that process. Details here: https://github.com/Crell/fp/blob/master/src/composition.php https://github.com/Crell/fp/blob/master/src/array.php Of particular note: 1. Because of PHP's inconsistent handling of excess arguments to functions, there MUST be separate versions of every function that takes a callback, one that passes the key and one that does not. It would be a fatal design flaw to do otherwise. Yes, this balloons the number of such functions, which sucks, but that's PHP for you. 2. There are ample use cases for most operations to return an array or a lazy iterable. Both totally exist. I solved that by also having a separate version of each function, eg, amap() vs itmap(). The former returned an array, the latter returned a generator that generated the equivalent array. It would be a fatal design flaw to not account for this. Yes, this balloons the number of such functions, which sucks, but that's PHP for you. So, eg, I have *four* map functions: amap(), itmap(), amapWithKeys(), itmapWithKeys(). Same for filter. Other operations only needed 2 variants, eg, first() and firstWithKeys(), any() and anyWithKeys(), etc. I do not claim that naming pattern to be ideal; in fact I don't particularly like *WithKeys(). We should think carefully on the naming. A possible alternative would be to always return a lazy iterable in all circumstances and assume someone can use to_array() or equivalent on the result if they want it as an array. (That's effectively what Python 3 does with comprehensions.) However, that could have non-trivial performance impact since generators are slower than plain arrays. 3. Feel free to borrow liberally, design-wise, from the above code. There's a few more methods in there that could be of use, too. Note, though, that all are designed to be used with a pipe(), so they mostly return a closure that has been manually partially applied with everything except the iterable, so you get a single-argument function, which is what a pipe() or compose() chain needs. Third, speaking of pipe, I disagree with Tim that putting the callback first would be easier for pipe/partials. If we ever get partials similar to previously implemented, then the argument order won't matter. If we get pipes as I've previously proposed, then none of these functions are directly usable because they're multi-argument. The alternative I've considered is somewhat inspired by Elixir (assuming I understand the little Elixir I've read), in which a function after a |> is automatically assumed to be partially applying everything but the first argument. So $list |> map($callable) translates to map($list, $callable). I've not decided yet if that's a good way to avoid needing full partial application or a good way to make horribly confusing code. But if that were to happen, it would only work if all of these functions took the iterable, the "object to be operated on", as their first argument. The callable, if inlined, is almost always the longest argument. That means it is most readable when it is the last argument, so there is no need to look at the end of the closure to see if there's any other arguments. (This is a problem with array_map() currently.) So I would instead propose that *all* iterator functions follow the pattern: name($iterable, other stuff, $callback_if_applicable); That is easily learnable, most likely to result in clean-ish code, and most likely to be nice with any future pipe or partial implementations. At worst, it would make pipe-ifying all such functions a trivially identical operation for all of them, making my library little more than a series of boring one-liners. (Please make my library little more than a series of boring one-liners.) That does also mean we cannot support variadics or optional arguments. I am OK with that. And if someone really needs a different order, well, we have named arguments now. Tim noted nesting these functions and what would make that cleanest. What would make it cleanest is to not nest them and instead use proper chaining instead; my pipe() function, a native pipe operator, or similar. Expecting these functions to nest and not be ugly is a fools errand, especially when there are vastly better options readily available. Fourth, I agree with Levi that figuring out the edge case handling around empty lists is crucial. The more we can design the sematics such that they "fall out" naturally, the better. Eg, first() may return null for not found, which dovetails nicely with the ?? operator to provide a default. However, that means null cannot be used as a meaningful found-value. I'd argue that is *the correct behavior*, but I'm sure some would disagree. An Option type would be nice, but to get that we really need to get ADTs first, and I don't have a timeline on that. Ilija is more interested in fixing core bugs right now than in adding new features, the silly man... :-P (Technically an Option/Maybe object could be implemented with just classes as we have them now, especially if it's done in core, but it would be cleaner and more ergonomic if built on top of an Enum.) Also, Monads are clunky in a language without first-class support for them. I've written extensively on this topic recently: https://peakd.com/hive-168588/@crell/much-ado-about-null I'm not sure of the best way forward here, other than it should be addressed very carefully and explicitly. Fifth, I would absolutely include map and filter in the included operations. They are critical parts of list handling. If we had pipe-compatible map and filter, that would basically give us a list comprehension tool for free. (That's exactly how many languages approach list comprehensions.) In my own list-operation-centric work, I've used map and filter a lot more than any of the other operations in the list above. I'm on board with the direction, modulo implementation details. --Larry Garfield