[RFC] 64 bit platform improvements for string length and integer

php.internals

Anatol Belski

12 years ago
Hi, I would sincerely invite everyone to the resurrected str_size_and_int64 RFC discussion. I propose the discussion to last for one week as allowed by the voting RFC because this topic has already been discussed to death previously (any objections?). As no userland change is introduced, the discussion would end and the voting would start on May 13th with 50%+1 votes requirement. https://wiki.php.net/rfc/size_t_and_int64_next Best regards Anatol

Andrew Faulds

12 years ago
On 6 May 2014, at 09:01, Anatol Belski <anatol.php@belski.net> wrote:
> https://wiki.php.net/rfc/size_t_and_int64_next
Am I reading this incorrectly, or is this suggesting switching to 64-bit types for int(), but only for 64-bit builds? I would prefer to switch to 64-bit on 32-bit platforms as well, for the sake of consistency. PHP code should behave identically on 32-bit and 64-bit systems.
-- Andrea Faulds http://ajf.me/

Derick Rethans

12 years ago
On Tue, 6 May 2014, Andrea Faulds wrote:
> > On 6 May 2014, at 09:01, Anatol Belski <anatol.php@belski.net> wrote: > > > https://wiki.php.net/rfc/size_t_and_int64_next > > Am I reading this incorrectly, or is this suggesting switching to > 64-bit types for int(), but only for 64-bit builds? I would prefer to > switch to 64-bit on 32-bit platforms as well, for the sake of > consistency. PHP code should behave identically on 32-bit and 64-bit > systems.
I would agree to that. And to counteract "this is going to be slower", well, perhaps, but then again, AMD64 processors have been out for more than 10 years. And there are even ARM64 processors now, and you'd hardly use ARM processors for running PHP at full speed.. If you're still on 32bit, maybe it's time to upgrade. cheers, Derick
-- http://derickrethans.nl | http://xdebug.org Like Xdebug? Consider a donation: http://xdebug.org/donate.php twitter: @derickr and @xdebug Posted with an email client that doesn't mangle email: alpine

Anatol Belski

12 years ago
Hi Andrea, On Tue, May 6, 2014 10:07, Andrea Faulds wrote:
>
> On 6 May 2014, at 09:01, Anatol Belski <anatol.php@belski.net> wrote: > > >> https://wiki.php.net/rfc/size_t_and_int64_next >> > > Am I reading this incorrectly, or is this suggesting switching to 64-bit > types for int(), but only for 64-bit builds? I would prefer to switch to > 64-bit on 32-bit platforms as well, for the sake of consistency. PHP code > should behave identically on 32-bit and 64-bit systems. >
this was not implemented. The complexity of the patch would increase as well as the danger of the miscarriage danger. You can see this as a middle stage, without this change one will have to start over for int64 on 32 bit. Whereby I have to mention that there's no plan to implement this yet, please note also the performance concerns. It's to see there are more people interested on that, if such a project exist later, i would participate on that. Regards Anatol

Pierre Joye

12 years ago
On Tue, May 6, 2014 at 10:01 AM, Anatol Belski <anatol.php@belski.net> wrote:
> Hi, > > I would sincerely invite everyone to the resurrected str_size_and_int64 > RFC discussion. > > I propose the discussion to last for one week as allowed by the voting RFC > because this topic has already been discussed to death previously (any > objections?). As no userland change is introduced, the discussion would > end and the voting would start on May 13th with 50%+1 votes requirement. > > https://wiki.php.net/rfc/size_t_and_int64_next
Before someone jumps to fast conclusions, we will obviously work with the phpng developers to figure out the best way to get these two proposals in core in a smooth and conflict-less way. However we really want to get a decision as it costs us quite some time to maintain this branch outside the main development streams while still working on all other parts of php.
-- Pierre @pierrejoye | http://www.libgd.org

Derick Rethans

12 years ago
On Tue, 6 May 2014, Anatol Belski wrote:
> I would sincerely invite everyone to the resurrected > str_size_and_int64 RFC discussion. > > I propose the discussion to last for one week as allowed by the voting > RFC because this topic has already been discussed to death previously > (any objections?). As no userland change is introduced, the discussion > would end and the voting would start on May 13th with 50%+1 votes > requirement. > > https://wiki.php.net/rfc/size_t_and_int64_next
I see this still says "The usage of long datatype continues on 32 bit platforms." — do I understand correctly that on 32bit platforms, there won't still be a 64bit integer in PHP and that this "only" makes 64bit integers work on the Windows LLP64 model? Or are they still 32bit integers there? Under "Accepting values with zend_parse_parameters()" “l” “i” to accept integer argument, the internal var has to be declared as php_int_t (inside PHP) or zend_int_t (inside Zend) “L” “I” to accept integer argument with range check, the internal var has to be declared as php_int_t (inside PHP) or zend_int_t (inside Zend) Using l and I is going to be major confusing and incredibly hard to spot. "'l', 'L', 's', 'p' parameter formats aren't available anymore" That means that every zpp call now needs to have an #ifdef around it to support post-this-patch and pre-this-patch PHP versions, as well as around their variable declarations. I'm afraid that will result in a huge mess, so we should look at this again. Under "Example on accepting parameters with zpp" You have: zend_parse_parameters(ZEND_NUM_ARGS() TSRMLS_CC, "iISP" But why is both i and I allowed (or did I misread the Old/New before?) If so, then my comment on i, l, I is already valid :-) Under "Example on printf specs usage" You have: php_error_docref(NULL TSRMLS_CC, E_WARNING, "Value '" ZEND_INT_FMT "' is out of range", i0); Using that macro there feels really unnatural, and many C developers will not bother to figure out that they use that, of course ,because they are familiar with "%d" and "%l". And similarly under "Example on printf specs usage (no BC)" for: "Value '%pd' is out of range", i0 The new macros that you use in the example for "char *dup_substr(zval *s, zval *i)" also infer that all those accesses need to have a big #ifdef around stuff to support multiple versions. Again, making it a major pain for extension developers to support pre- and post-patch PHP versions. I'm afraid with all those concerns, I will have to vote "no". cheers, Derick
-- http://derickrethans.nl | http://xdebug.org Like Xdebug? Consider a donation: http://xdebug.org/donate.php twitter: @derickr and @xdebug Posted with an email client that doesn't mangle email: alpine

Anatol Belski

12 years ago
Hi Derick, On Tue, May 6, 2014 11:13, Derick Rethans wrote:
> On Tue, 6 May 2014, Anatol Belski wrote: > > >> I would sincerely invite everyone to the resurrected >> str_size_and_int64 RFC discussion. >> >> I propose the discussion to last for one week as allowed by the voting >> RFC because this topic has already been discussed to death previously >> (any objections?). As no userland change is introduced, the discussion >> would end and the voting would start on May 13th with 50%+1 votes >> requirement. >> >> https://wiki.php.net/rfc/size_t_and_int64_next >> > > I see this still says "The usage of long datatype continues on 32 bit > platforms." — do I understand correctly that on 32bit platforms, there > won't still be a 64bit integer in PHP and that this "only" makes 64bit > integers work on the Windows LLP64 model? Or are they still 32bit integers > there? >
Yes, this adds int64 to the windows builds and size_t everywhere and hence makes it work consistent all 64 bit platforms. Regarding int64 on 32 bit platform - no, this is not implemented. There are various reasons for that, the main reason here - the complexity. To the time of starting such a huge thing the danger it to be not finished is big. You can see this as a stage which makes int64 on 32 bit platforms at least hypotetically possible. That might be not visible well to the external APIs, but internally using int64 on 32 bit platforms would at least double the complexity of the patch itself (which is already not a light one). Also, without it we can't move further with overall improvement. And we can't achieve everything in one single patch, obviously.
> Under "Accepting values with zend_parse_parameters()" > > > “l” “i” to accept integer argument, the internal var has to be declared > as php_int_t (inside PHP) or zend_int_t (inside Zend) “L” “I” to accept > integer argument with range check, the internal var has to be declared as > php_int_t (inside PHP) or zend_int_t (inside Zend) > > Using l and I is going to be major confusing and incredibly hard to > spot. > > "'l', 'L', 's', 'p' parameter formats aren't available anymore" > > > That means that every zpp call now needs to have an #ifdef around it to > support post-this-patch and pre-this-patch PHP versions, as well as around > their variable declarations. I'm afraid that will result in a huge mess, > so we should look at this again. > > Under "Example on accepting parameters with zpp" > > > You have: > zend_parse_parameters(ZEND_NUM_ARGS() TSRMLS_CC, "iISP" > > But why is both i and I allowed (or did I misread the Old/New before?) > If so, then my comment on i, l, I is already valid :-) >
The syntax shown there is pure new syntax. Please take a look at the migration stuff here http://git.php.net/?p=php-src.git;a=tree;f=compat;hb=refs/heads/str_size_and_int64 (also mentioned in the RFC). It's relatively fresh yet, but already offers some automatic and will definitely get better on and on. The compat.h header defines ZPP_FMT_COMPAT macros, also there's the clang zpp checker which will reveal any incompatibility. I think you misread the "big I" with "small l" which can happen also another way round :)
> Under "Example on printf specs usage" > > > You have: > php_error_docref(NULL TSRMLS_CC, E_WARNING, "Value '" ZEND_INT_FMT "' is > out of range", i0); > > Using that macro there feels really unnatural, and many C developers > will not bother to figure out that they use that, of course ,because they > are familiar with "%d" and "%l". > > And similarly under "Example on printf specs usage (no BC)" for: > > > "Value '%pd' is out of range", i0 > > > The new macros that you use in the example for "char *dup_substr(zval > *s, zval *i)" also infer that all those accesses need to have a big > #ifdef around stuff to support multiple versions. Again, making it a > major pain for extension developers to support pre- and post-patch PHP > versions. >
For this two I would mention the migration path as well. ZEND_INT_FMT is exactly for the purpose to output zend_int_t be it 64 or 32 bit and care about BC. It will define to the right format for the older PHP through compat.h. The %d is kept for the case of the pure 32 bit int usage. Previously it is %ld but it couldn't work anymore in the new code as it would mean different things with consistent int64 support. The %pd variant however can be used in the developments without need of BC. With the new macros names - they're all aliased in the compat.h for the older versions. So the general idea of the migration path is the usage of the new clean syntax for the development (like calling a cat a cat) and retaining the compat layer to the older versions. Once migrated, that can be carried on.
> I'm afraid with all those concerns, I will have to vote "no". >
Of course that would make me sad. Not only for the work done on that, but also for the future perspectives and for the quantity of the users who wish such thing. Cheers Anatol

Lester Caine

12 years ago
On 06/05/14 11:58, Anatol Belski wrote:
>> I'm afraid with all those concerns, I will have to vote "no". >> > > Of course that would make me sad. Not only for the work done on that, but > also for the future perspectives and for the quantity of the users who > wish such thing.
Having just been going through an exercise to convert Windows XP sites to something M$ will allow to run without warning messages, the switch to a more modern windows would seem sensible. Except we have been restricted to using 32bit builds of windows for compatibility with the hardware. While parts of PHP already require 64 bit integers which are handled in an even less efficient way, switching to a clean 64 bit integer across all platforms really is essential. Having to manage the difference in the user code base is simply not an option :( The switch to a cleaner implementation base should allow proper planning for that requirement?
-- Lester Caine - G8HFL ----------------------------- Contact - http://lsces.co.uk/wiki/?page=contact L.S.Caine Electronic Services - http://lsces.co.uk EnquirySolve - http://enquirysolve.com/ Model Engineers Digital Workshop - http://medw.co.uk Rainbow Digital Media - http://rainbowdigitalmedia.co.uk

Anatol Belski

12 years ago
Hi Lester, On Tue, May 6, 2014 15:14, Lester Caine wrote:
> On 06/05/14 11:58, Anatol Belski wrote: > >>> I'm afraid with all those concerns, I will have to vote "no". >>> >>>> >> Of course that would make me sad. Not only for the work done on that, >> but also for the future perspectives and for the quantity of the users >> who wish such thing. > > Having just been going through an exercise to convert Windows XP sites > to something M$ will allow to run without warning messages, the switch to a > more modern windows would seem sensible. Except we have been restricted to > using 32bit builds of windows for compatibility with the hardware. While > parts of PHP already require 64 bit integers which are handled in an even > less efficient way, switching to a clean 64 bit integer across all > platforms really is essential. Having to manage the difference in the user > code base is simply not an option :( > > The switch to a cleaner implementation base should allow proper planning > for that requirement? >
absolutely, furthermore - Windows Server 2012 (and probably any upcoming) is not merchandased with 32-bit edition anymore. For 32 bit PHP there it means possible performance penalty. So usage of the true 64 bit PHP fully makes sense there. Regards Anatol

Christopher Jones

12 years ago
On 5/6/14, 1:01 AM, Anatol Belski wrote:
> Hi, > > I would sincerely invite everyone to the resurrected str_size_and_int64 > RFC discussion. > > I propose the discussion to last for one week as allowed by the voting RFC > because this topic has already been discussed to death previously (any > objections?). As no userland change is introduced, the discussion would > end and the voting would start on May 13th with 50%+1 votes requirement. > > https://wiki.php.net/rfc/size_t_and_int64_next > > Best regards > > Anatol >
It seems the "shiny" phpng is attracting all the attention and you are unlikely to get much discussion: we had so much last time anyway. Can you add the proposed voting options to the RFC? Also if you can resolve the open issue on dead SAPIs first, that would be great. Then lets vote. Chris
-- christopher.jones@oracle.com http://twitter.com/ghrd Free PHP & Oracle book: http://www.oracle.com/technetwork/topics/php/underground-php-oracle-manual-098250.html

Anatol Belski

12 years ago
Hi Chris, On Wed, May 7, 2014 19:32, Christopher Jones wrote:
>
> > On 5/6/14, 1:01 AM, Anatol Belski wrote: > >> Hi, >> >> >> I would sincerely invite everyone to the resurrected str_size_and_int64 >> RFC discussion. >> >> >> I propose the discussion to last for one week as allowed by the voting >> RFC >> because this topic has already been discussed to death previously (any >> objections?). As no userland change is introduced, the discussion would >> end and the voting would start on May 13th with 50%+1 votes >> requirement. >> >> https://wiki.php.net/rfc/size_t_and_int64_next >> >> >> Best regards >> >> >> Anatol >> >> > > It seems the "shiny" phpng is attracting all the attention and you are > unlikely to get much discussion: we had so much last time anyway. > > Can you add the proposed voting options to the RFC? Also if you can > resolve the open issue on dead SAPIs first, that would be great. Then lets > vote. >
Actually this RFC is being even partly discussed in the parallel phpng thread and on IRC :) These two RFCs definitely intersect and are probably the most significant changes proposed for PHP next. As the voting rule says the discussion "should be at least a week", IMHO lets keep it then (not long till 13th anyway). The vote option would be a simple yes/no choice to accept the RFC for the next major PHP version, just added it. For the SAPI RFC - it's probably not to be solved in such short term of a couple of days (and that's why it was separated from the main one). As it involves trying all the corresponding servers, mailing the authors and waiting for their reaction. Now it also looks like this topic concerns phpng as well. Thus it makes sense firstly to finish the essential change. The task itself of the research and adjoining throw out is a routine. Best Anatol