Everything

php

[RFC] Add num_available_processors

php.internals

Daniel Kesselberg

1 year ago

Hi everyone,

I'm happy to share my first RFC :) It proposes adding a small function
to retrieve the number of available processors; a feature that's
commonly found in other programming languages and one that I believe
would be a useful addition to PHP.

The related PR has already received a bit of early traction, and now
that the RFC is complete, I'm looking forward to your feedback!

RFC: https://wiki.php.net/RFC/num_available_processors
Patch: https://github.com/php/php-src/pull/11137

Best
Daniel

Niels Dossche

1 year ago

Hi

In my opinion, the return type should not be nullable.
Returning NULL when the platform (or PHP on that platform) doesn't support getting this information is an anti-pattern.
Instead, availability of the necessary functionality should be checked at configure time and the function should be made conditionally available.
That way, the return type can just be "int".

Kind regards
Niels

Rob Landers

1 year ago

On Sat, May 24, 2025, at 19:42, Niels Dossche wrote:

> Hi
>
> In my opinion, the return type should not be nullable.
> Returning NULL when the platform (or PHP on that platform) doesn't support getting this information is an anti-pattern.
> Instead, availability of the necessary functionality should be checked at configure time and the function should be made conditionally available.
> That way, the return type can just be "int".
>
> Kind regards
> Niels
>

I’m curious why you say this is an “anti pattern”? I do agree that it should return a number or throw. There are various error conditions it should throw (at least on Linux) so having it throw an exception when there isn’t a way to count any processors makes more sense than returning null.

I also note the patch is using the sysconf method and not checking the error on a negative one result and not properly handling zero. Naturally, this means something might be catastrophically wrong, but it should still throw instead of returning null. It could also be someone patching libc to get around per-core licensing too. I’ve seen the latter more often than the former.

— Rob

Niels Dossche

1 year ago

On 24/05/2025 21:24, Rob Landers wrote:

>
>
> On Sat, May 24, 2025, at 19:42, Niels Dossche wrote:
>> Hi
>>
>> In my opinion, the return type should not be nullable.
>> Returning NULL when the platform (or PHP on that platform) doesn't support getting this information is an anti-pattern.
>> Instead, availability of the necessary functionality should be checked at configure time and the function should be made conditionally available.
>> That way, the return type can just be "int".
>>
>> Kind regards
>> Niels
>>
>
> I’m curious why you say this is an “anti pattern”? I do agree that it should return a number or throw.

If you make the function unconditionally available, yet specifying the return type as ?int, then you give the false impression that it _can_ work even if your platform doesn't support it.

Also: IMO new APIs should fail hard with an exception if they can't do their main task, that's our error "channel".

> There are various error conditions it should throw (at least on Linux) so having it throw an exception when there isn’t a way to count any processors makes more sense than returning null.

Throwing is indeed preferable.
What are the error conditions on Linux?

>
> It could also be someone patching libc to get around per-core licensing too. I’ve seen the latter more often than the former. 

I don't follow.
In any case, if you run a monkey patched libc and you're breaking the consumer expectations of said libc, then you deserve getting it blown up in your face imo.

Kind regards
Niels

Rob Landers

1 year ago

On Sat, May 24, 2025, at 22:28, Niels Dossche wrote:

> On 24/05/2025 21:24, Rob Landers wrote:
> >
> >
> > On Sat, May 24, 2025, at 19:42, Niels Dossche wrote:
> >> Hi
> >>
> >> In my opinion, the return type should not be nullable.
> >> Returning NULL when the platform (or PHP on that platform) doesn't support getting this information is an anti-pattern.
> >> Instead, availability of the necessary functionality should be checked at configure time and the function should be made conditionally available.
> >> That way, the return type can just be "int".
> >>
> >> Kind regards
> >> Niels
> >>
> >
> > I’m curious why you say this is an “anti pattern”? I do agree that it should return a number or throw.
>
> If you make the function unconditionally available, yet specifying the return type as ?int, then you give the false impression that it _can_ work even if your platform doesn't support it.
>
> Also: IMO new APIs should fail hard with an exception if they can't do their main task, that's our error "channel".
>
> > There are various error conditions it should throw (at least on Linux) so having it throw an exception when there isn’t a way to count any processors makes more sense than returning null.
>
> Throwing is indeed preferable.
> What are the error conditions on Linux?
>
> >
> > It could also be someone patching libc to get around per-core licensing too. I’ve seen the latter more often than the former.
> I don't follow.
> In any case, if you run a monkey patched libc and you're breaking the consumer expectations of said libc, then you deserve getting it blown up in your face imo.
>
> Kind regards
> Niels
>

I think we both agree it should blow up.

Error conditions (for Linux) using the current method in the patch are either -1 or 0. -1 is just a possible return for that function which could indicate that the kernel doesn’t have support for that sysconf value or there are infinite cores — you’d have to check errno.

It might also report 0, which technically isn’t an error (hotplugging cpus for instance). It could also just be a bug. This is a fun one: https://github.com/lxc/lxcfs/issues/469

Also, I wouldn’t be surprised to see a container with very small amounts of cpu allocated getting rounded to zero. But in any case, zero should probably be treated as an error IMHO.

— Rob

Ben Ramsey

1 year ago

On 5/24/25 15:28, Niels Dossche wrote:

> On 24/05/2025 21:24, Rob Landers wrote:
>>
>>
>> On Sat, May 24, 2025, at 19:42, Niels Dossche wrote:
>>> Hi
>>>
>>> In my opinion, the return type should not be nullable.
>>> Returning NULL when the platform (or PHP on that platform) doesn't support getting this information is an anti-pattern.
>>> Instead, availability of the necessary functionality should be checked at configure time and the function should be made conditionally available.
>>> That way, the return type can just be "int".
>>>
>>> Kind regards
>>> Niels
>>>
>>
>> I’m curious why you say this is an “anti pattern”? I do agree that it should return a number or throw.
>
> If you make the function unconditionally available, yet specifying the return type as ?int, then you give the false impression that it _can_ work even if your platform doesn't support it.
>
> Also: IMO new APIs should fail hard with an exception if they can't do their main task, that's our error "channel".
>
>> There are various error conditions it should throw (at least on Linux) so having it throw an exception when there isn’t a way to count any processors makes more sense than returning null.
>
> Throwing is indeed preferable.
> What are the error conditions on Linux?

Are you suggesting that the function itself not be available or that it
throws when you attempt to call it on a system that doesn't support it?

I don't like the idea of the function not being available if the system
doesn't support the functionality. Users would have to call
function_exists() before using the function, in those cases.

Random thought: if the system doesn't support the functionality, would
it be wrong to respond with a default value of 1 or 0? Presumably, all
systems have at least 1 core (maybe that's a naïve assumption). (I'm not
in favor of this approach—it's just a random thought.)

IMO, it's best to throw an exception when calling the function if the
system doesn't support it.

Cheers,
Ben

Derick Rethans

1 year ago

On 27 May 2025 17:37:47 BST, Ben Ramsey <ramsey@php.net> wrote:

>
>Are you suggesting that the function itself not be available or that it throws when you attempt to call it on a system that doesn't support it?
>
>I don't like the idea of the function not being available if the system doesn't support the functionality. Users would have to call function_exists() before using the function, in those cases.

This has been the accepted practise for years, if not decades.

If functionality isn't available, the function shouldn't exist. We do the same for posix (such as posix_getrlimit) and DNS related functions.

This also allows for polyfills.

cheers
Derick

Ben Ramsey

1 year ago

On 5/27/25 12:01, Derick Rethans wrote:

> On 27 May 2025 17:37:47 BST, Ben Ramsey <ramsey@php.net> wrote:
>>
>> Are you suggesting that the function itself not be available or that it throws when you attempt to call it on a system that doesn't support it?
>>
>> I don't like the idea of the function not being available if the system doesn't support the functionality. Users would have to call function_exists() before using the function, in those cases.
>
>
> This has been the accepted practise for years, if not decades.
>
> If functionality isn't available, the function shouldn't exist. We do the same for posix (such as posix_getrlimit) and DNS related functions.
>
> This also allows for polyfills.
>
> cheers
> Derick

I see. Carry on, then. :-)

Cheers,
Ben

Daniel Kesselberg

1 year ago

Hi,

Thanks for all your feedback on the RFC.

I've updated the RFC to incorporate most of your feedback:
https://wiki.php.net/rfc/num_available_processors

1) The limitation, that the CPU affinity mask is ignored
2) The naming discussion
3) Function availability on unsupported platforms
4) Return type

How do we continue? ;) I see there are various ideas how to approach it,
is that something you would vote (let's do a or b) on, or how does that
work?

Best
Daniel

On 2025-05-27 19:13, Ben Ramsey wrote:

Tim Düsterhus

1 year ago

Hi

Am 2025-06-15 19:30, schrieb Daniel Kesselberg:

> How do we continue? ;) I see there are various ideas how to approach
> it, is that something you would vote (let's do a or b) on, or how does
> that work?

Having secondary votes is generally frowned upon for this kind of simple
RFCs. It's mostly used when there are multiple equally-valid choices
(which I don't think is the case here). As the RFC author you are
expected to be the subject-matter expert and should make a decision that
is then voted on, rather than deciding every detail by an individual
vote.

Best regards
Tim Düsterhus

Gina P. Banyard

1 year ago

On Sunday, 15 June 2025 at 19:32, Daniel Kesselberg <mail@danielkesselberg.de> wrote:

> Hi,
>
> Thanks for all your feedback on the RFC.
>
> I've updated the RFC to incorporate most of your feedback:
> https://wiki.php.net/rfc/num_available_processors
>
> 1) The limitation, that the CPU affinity mask is ignored

This is something you need to decide on as the RFC author.

> 2) The naming discussion

I don't care much about this topic, but possibly adding the sys_ prefix makes the most sense to "soft namespace" the function.

> 3) Function availability on unsupported platforms

The function should not be available if it is unsupported,
as this how every other conditionally supported feature is handled.

> 4) Return type

Arguably, this should always return an integer.
Which if my understanding is correct, when you make it conditionally available then it cannot fail.

> How do we continue? ;) I see there are various ideas how to approach it,
> is that something you would vote (let's do a or b) on, or how does that
> work?

Small RFCs should only have a single yes/no vote.
As the RFC author, you can (and should) be opinionated.

Best regards,

Gina P. Banyard

PS: Minor list etiquette remainder, please do not top post, and only keep the relevant things while quoting.

Eric Norris

1 year ago

Hello!

On Sun, Jun 15, 2025 at 1:33 PM Daniel Kesselberg
<mail@danielkesselberg.de> wrote:

>
> Hi,
>
> Thanks for all your feedback on the RFC.
>
> I've updated the RFC to incorporate most of your feedback:
> https://wiki.php.net/rfc/num_available_processors
>
> 1) The limitation, that the CPU affinity mask is ignored
> 2) The naming discussion

I'm curious, is there a reason why you feel that introducing a new
namespace should be under "Future Scope", instead of a part of this
RFC? I expect there would be some bikeshedding over the namespace
name, but there is / will be some bikeshedding around a non-namespaced
name in the first place, so it feels like a good opportunity to
embrace namespaces in the standard library.

I ask this because I've been very slightly interested in introducing a
Curl namespace since the "Throwable Hierarchy Policy for Extensions"
(https://wiki.php.net/rfc/extension_exceptions) RFC and my own
personal "Persistent curl share handle"
(https://wiki.php.net/rfc/curl_share_persistence_improvement) RFC.

> How do we continue? ;) I see there are various ideas how to approach it,
> is that something you would vote (let's do a or b) on, or how does that
> work?

Relating to my above question, if you're open to it, I would be happy
to work with you on your RFC to (a) figure out a good namespace name
and (b) update the RFC to match.

Côme Chilliet

1 year ago

Le samedi 24 mai 2025, 19:42:34 heure d’été d’Europe centrale Niels Dossche a écrit :

> In my opinion, the return type should not be nullable.
> Returning NULL when the platform (or PHP on that platform) doesn't support getting this information is an anti-pattern.
> Instead, availability of the necessary functionality should be checked at configure time and the function should be made conditionally available.
> That way, the return type can just be "int".

I strongly disagree here, having PHP function not always available is a mess, and forces function_exists checks all over the code. It feels archaic.

I get the idea of throwing instead of returning NULL for errors, but the idea behind the nullable return is to get this nice thing:
$cpus = num_available_cpus() ?? 1;

If the function throws on error, this gets more convoluted. Maybe it can be replaced by a parameter $defaultNumber and only throw when no default is given?

Regarding the CPUs available for PHP or total number discussion, maybe that justifies the "available" keyword in the name and it returns only those available to PHP, and we add a second function for total number?
Or as some people suggested in the PR, we add a namespaced class with methods for this.

Côme

Rob Landers

1 year ago

On Sat, May 24, 2025, at 19:37, Daniel Kesselberg wrote:

> Hi everyone,
>
> I'm happy to share my first RFC :) It proposes adding a small function
> to retrieve the number of available processors; a feature that's
> commonly found in other programming languages and one that I believe
> would be a useful addition to PHP.
>
> The related PR has already received a bit of early traction, and now
> that the RFC is complete, I'm looking forward to your feedback!
>
> RFC: https://wiki.php.net/RFC/num_available_processors
> Patch: https://github.com/php/php-src/pull/11137
>
> Best
> Daniel
>

Looks good!

My main question is: what is this actually counting? In the RFC it mentions "available processing units" ... which means, what? What counts as a "processing unit"? Are we talking about CPU Threads/cores; NPU cores; TPM cores; clocks? GPS? GPU? ... a modern computer has many "processing units" for different purposes and workloads. I’m assuming this is CPU Threads, not physical cores? I will refer to CPU Threads as "Logical Cores" so we all don’t get confused since most of us here are programmers and saying "thread" has a different meaning.

Secondly, how is it counting "available"? If I assign PHP to a specific CPU affinity mask (say one logical core), will it return 1, or the total number of logical cores available on my machine? I would expect it to be 1, since PHP only has access to 1, but I can also see the logic in returning the total number.

— Rob

Alwin Garside

1 year ago

On 24 May 2025, at 20:48, Rob Landers <rob@bottled.codes> wrote:

>
> On Sat, May 24, 2025, at 19:37, Daniel Kesselberg wrote:
>>
>>
>> Hi everyone,
>>
>> I'm happy to share my first RFC :) It proposes adding a small function
>> to retrieve the number of available processors; a feature that's
>> commonly found in other programming languages and one that I believe
>> would be a useful addition to PHP.
>>
>> The related PR has already received a bit of early traction, and now
>> that the RFC is complete, I'm looking forward to your feedback!
>>
>> RFC: https://wiki.php.net/RFC/num_available_processors
>> Patch: https://github.com/php/php-src/pull/11137
>>
>> Best
>> Daniel
>>
>
> Looks good!
>
> My main question is: what is this actually counting? In the RFC it mentions "available processing units" ... which means, what? What counts as a "processing unit"? Are we talking about CPU Threads/cores; NPU cores; TPM cores; clocks? GPS? GPU? ... a modern computer has many "processing units" for different purposes and workloads. I’m assuming this is CPU Threads, not physical cores? I will refer to CPU Threads as "Logical Cores" so we all don’t get confused since most of us here are programmers and saying "thread" has a different meaning.
>
> Secondly, how is it counting "available"? If I assign PHP to a specific CPU affinity mask (say one logical core), will it return 1, or the total number of logical cores available on my machine? I would expect it to be 1, since PHP only has access to 1, but I can also see the logic in returning the total number.
>
> — Rob

Hi Daniel,

I agree with Rob that "processor" is a bit too ambiguous. I'd use the phrase "cpu_core" instead. Yes, technically that's not entirely accurate when hyper-threading is used, but in most cases it's not trivial to distinguish physical cores from logical cores anyway, and "cpu_cores" provides the most understandable abstraction for the vast majority of use cases: deciding how many parallel processes one should use for optimal use of the CPU.

Also, is it really necessary to add "available" as a disambiguator? In other words: are there future plans to add a function that provides the "total" or "unavailable" number of processors? If not, I'd just drop the "available" part.

Finally, from a quick search in php-src there doesn't seem to be any existing function name that starts with `num_`. For the sake of consistency with the existing PHP functions, and similar functionality in other languages, I suggest suffixing the function name with `_count` instead.

So, to wrap this all up, I'd like to respectfully propose the following function name instead:

cpu_core_count()

Alwin

Rob Landers

1 year ago

On Sun, May 25, 2025, at 12:07, Alwin Garside wrote:

> On 24 May 2025, at 20:48, Rob Landers <rob@bottled.codes> wrote:
> >
> > On Sat, May 24, 2025, at 19:37, Daniel Kesselberg wrote:
> >>
> >>
> >> Hi everyone,
> >>
> >> I'm happy to share my first RFC :) It proposes adding a small function
> >> to retrieve the number of available processors; a feature that's
> >> commonly found in other programming languages and one that I believe
> >> would be a useful addition to PHP.
> >>
> >> The related PR has already received a bit of early traction, and now
> >> that the RFC is complete, I'm looking forward to your feedback!
> >>
> >> RFC: https://wiki.php.net/RFC/num_available_processors
> >> Patch: https://github.com/php/php-src/pull/11137
> >>
> >> Best
> >> Daniel
> >>
> >
> > Looks good!
> >
> > My main question is: what is this actually counting? In the RFC it mentions "available processing units" ... which means, what? What counts as a "processing unit"? Are we talking about CPU Threads/cores; NPU cores; TPM cores; clocks? GPS? GPU? ... a modern computer has many "processing units" for different purposes and workloads. I’m assuming this is CPU Threads, not physical cores? I will refer to CPU Threads as "Logical Cores" so we all don’t get confused since most of us here are programmers and saying "thread" has a different meaning.
> >
> > Secondly, how is it counting "available"? If I assign PHP to a specific CPU affinity mask (say one logical core), will it return 1, or the total number of logical cores available on my machine? I would expect it to be 1, since PHP only has access to 1, but I can also see the logic in returning the total number.
> >
> > — Rob
>
>
> Hi Daniel,
>
> I agree with Rob that "processor" is a bit too ambiguous. I'd use the phrase "cpu_core" instead. Yes, technically that's not entirely accurate when hyper-threading is used, but in most cases it's not trivial to distinguish physical cores from logical cores anyway, and "cpu_cores" provides the most understandable abstraction for the vast majority of use cases: deciding how many parallel processes one should use for optimal use of the CPU.

Yes, that is why I was curious as to the method it was using to return the number. Using the sysconf method will just return the total number of available cores. However, in Linux there is also the sched_getaffinity method which returns the total number of cores allocated to the current process (which may be lower). For example, when sharing a webserver and a worker queue on a single system, I may only allocate 1/3 of my cores (via `taskset`) to the worker queue, and 1/3 cores to the webserver and the last 1/3 for other tasks (such as a database). This prevents the worker queue from taking over the entire system and slowing down my web requests and database.

If we have 12 cores and I’ve only allocated 4 cores for PHP, I would want this function to tell my worker queue that it only has 4 cores, using the system above — not 12. This probably doesn’t matter too much when this is the only process on the machine. However, in any serious production environment, you want to be able to prevent a run-away process/job from harming other unrelated aspects of the system.

You can also see this behaviour with nproc:

❯ taskset -c 0 nproc
1

❯ nproc
16

— Rob

Daniel Kesselberg

1 year ago

Hi Rob,

Thanks for your feedback. I've added a note, to the proposal section,
that the cpu affinity mask is currently not considered.

The current patch is rather small. Taking the CPU affinity mask into
account will require a more work and introduces different code paths for
linux, bsd and win.

I agree, it's more correct to take into account and wonder if it's
probably enough to just document for now. What do you think, is that
something that should go into the voting part (take cpu affinity mask
into account)?

RFC: https://wiki.php.net/rfc/num_available_processors

On 2025-05-25 15:28, Rob Landers wrote:

Tim Düsterhus

1 year ago

Hi

Am 2025-05-25 12:07, schrieb Alwin Garside:

> Finally, from a quick search in php-src there doesn't seem to be any
> existing function name that starts with `num_`. For the sake of
> consistency with the existing PHP functions, and similar functionality
> in other languages, I suggest suffixing the function name with `_count`
> instead.
>
> So, to wrap this all up, I'd like to respectfully propose the following
> function name instead:
>
>     cpu_core_count()

Naming is a good point indeed and one that I also hinted at with my
comment on the PR:
https://github.com/php/php-src/pull/11137#issuecomment-2888977523

I very strongly disagree with adding more “randomly named” functions to
ext/standard, the times of doing that are over. I would also claim that
it violates the “Coding Standards and Naming Policy”
(https://github.com/php/policies/blob/main/coding-standards-and-naming.rst)
in spirit.

At the very least the function should have a clear “prefix”. `sys` as in
`sys_getloadavg()` or `sys_get_temp_dir()` would probably work.
Alternatively `get` as in `getmyuid()` or `getrusage()` would also work.

As I've indicated in the PR, my preference would be creating a new
namespace and moving over the matching functions together with a
discussion of some namespace structure, including matching exceptions
(see https://wiki.php.net/rfc/extension_exceptions).

Best regards
Tim Düsterhus

Larry Garfield

1 year ago

On Sat, May 24, 2025, at 12:37 PM, Daniel Kesselberg wrote:

> Hi everyone,
>
> I'm happy to share my first RFC :) It proposes adding a small function
> to retrieve the number of available processors; a feature that's
> commonly found in other programming languages and one that I believe
> would be a useful addition to PHP.
>
> The related PR has already received a bit of early traction, and now
> that the RFC is complete, I'm looking forward to your feedback!
>
> RFC: https://wiki.php.net/RFC/num_available_processors
> Patch: https://github.com/php/php-src/pull/11137
>
> Best
> Daniel

I don't really have a problem with adding a function to expose this value, in concept.  But what is the use case, exactly?  The RFC mentions something about php-cs-fixer in passing in half a sentence, but doesn't explain it further.  Please explain further what value this has for core, rather than the parallels extension.  (In the RFC, not just here on list.)

--Larry Garfield

Daniel Kesselberg

1 year ago

Hi Larry,

Thanks for your feedback.

I've reworked the introduction at
https://wiki.php.net/rfc/num_available_processors and hopefully the use
case is a bit clearer now.

Best
Daniel

On 2025-05-25 22:22, Larry Garfield wrote:

Larry Garfield

1 year ago

On Sun, Jun 15, 2025, at 9:04 AM, Daniel Kesselberg wrote:

> Hi Larry,
>
> Thanks for your feedback.
>
> I've reworked the introduction at
> https://wiki.php.net/rfc/num_available_processors and hopefully the use
> case is a bit clearer now.
>
> Best
> Daniel

It is, thanks.

For the name, I don't like the idea of this being the first `num_` function.  This feels like a place we want to start a namespace for such system information functions, where we can then name it in a contextually-informed way.

--Larry Garfield