Reproducible Builds

php.internals

Sebastian Bergmann

2 years ago
I recently watched a video [1] that once again brought the topic of reproducible builds [2] to my attention. I believe that reproducible builds are becoming more and more important and that the build of the PHP interpreter/runtime should become reproducible. Right now, compiling the same version of PHP's C sources in the same environment (using the same compiler, against the same dependencies, etc.) produces a different binary every time. "Different" meaning that the built artifacts, the "php" executable for the CLI SAPI, for example, are not bit-by-bit identical. One obvious reason why this is the case is the fact that we use __DATE__ and __TIME__ in a couple of places. These preprocessor macros are expanded by the C compiler at compile-time to the current date and time. They are used in sapi/cli/php_cli.c, for instance, so that the output of "php -i" contains the date and time when the executable was compiled. I have not yet checked whether usage of the __DATE__ and __TIME__ macros is the only thing that makes the compilation of PHP irreproducible, but no longer using them would be a good start on the path towards reproducible builds. While we could probably replace __DATE__ and __TIME__ with SOURCE_DATE_EPOCH [3] [4], I cannot help but wonder whether having the date and time when the executable was built in the executable is actually useful. How attached are we to having the date and time of the build in the output of phpinfo(), "php -i", etc.? AFAIK, the topic of reproducible builds was brought up in 2017 for the first, and before this email only, time [5]. There was a PR [6] that was merged into PHP 7.1 which introduced the use of SOURCE_DATE_EPOCH to define PHP_BUILD_DATE in configure.ac. Today, when I grep for SOURCE_DATE_EPOCH on the master branch, I do not find any usage of SOURCE_DATE_EPOCH anymore. Or PHP_BUILD_DATE, for that matter.
-- [1] https://media.ccc.de/v/camp2023-57236-reproducible_builds_the_first_ten_years [2] https://reproducible-builds.org/ [3] https://reproducible-builds.org/specs/source-date-epoch/ [4] https://reproducible-builds.org/docs/source-date-epoch/ [5] https://externals.io/message/101327#101327 [6] https://github.com/php/php-src/pull/2965

Ilija Tovilo

2 years ago
Hi Sebastian On Tue, Nov 28, 2023 at 6:28 PM Sebastian Bergmann <sebastian@php.net> wrote:
> > I recently watched a video [1] that once again brought the topic of > reproducible builds [2] to my attention. > ... > I have not yet checked whether usage of the __DATE__ and __TIME__ macros > is the only thing that makes the compilation of PHP irreproducible, but no > longer using them would be a good start on the path towards reproducible > builds.
At least for core, enabled-by-default extensions, __DATE__ and __TIME__ seem to be the only variables. I can get reproducible builds by setting SOURCE_DATE_EPOCH.
> While we could probably replace __DATE__ and __TIME__ with > SOURCE_DATE_EPOCH [3] [4], ...
Both GCC and Clang support SOURCE_DATE_EPOCH out of the box, setting __DATE__ and __TIME__ accordingly. MSVC (shockingly) does not. However, reproducible builds likely don't matter as much for Windows since we provide the binaries for it. That said, I wouldn't object to removing the date either. Ilija

Marco Pivetta

2 years ago
On Tue, 28 Nov 2023 at 19:40, Ilija Tovilo <tovilo.ilija@gmail.com> wrote:
> That said, I wouldn't object to removing the date either. >
Wishful thinking, but perhaps a GIT ref of some sort would be a good replacement too, if the working copy is clean. I wouldn't put too much weight on it, but that would certainly help people while jumping across branches, when trying out new RFCs, and it should be stable. Marco Pivetta https://mastodon.social/@ocramius https://ocramius.github.io/

Sebastian Bergmann

2 years ago
Am 28.11.2023 um 19:40 schrieb Ilija Tovilo:
> At least for core, enabled-by-default extensions, __DATE__ and > __TIME__ seem to be the only variables. I can get reproducible builds > by setting SOURCE_DATE_EPOCH.
Confirmed: I can get reproducible builds, too, by using CLANG and setting SOURCE_DATE_EPOCH.

Derick Rethans

2 years ago
On 28 November 2023 17:28:18 GMT, Sebastian Bergmann <sebastian@php.net> wrote:
>While we could probably replace __DATE__ and __TIME__ with SOURCE_DATE_EPOCH [3] [4], I cannot help but wonder whether having the date and time when the executable was built in the executable is actually useful. How attached are we to having the date and time of the build in the output of phpinfo(), "php -i", etc.?
It is really useful for the development versions of PHP. Knowing whether your are running a PHP-dev from last week or last month is important. For released versions, not so much. cheers Derick

Matthew Weier O'Phinney

2 years ago
On Tue, Nov 28, 2023, 5:28 PM Derick Rethans <derick@php.net> wrote:
> On 28 November 2023 17:28:18 GMT, Sebastian Bergmann <sebastian@php.net> > wrote: > > >While we could probably replace __DATE__ and __TIME__ with > SOURCE_DATE_EPOCH [3] [4], I cannot help but wonder whether having the date > and time when the executable was built in the executable is actually > useful. How attached are we to having the date and time of the build in the > output of phpinfo(), "php -i", etc.? > > It is really useful for the development versions of PHP. Knowing whether > your are running a PHP-dev from last week or last month is important.
Would Marco's suggestion of using a git hash solve that? You'd then get both a reproducible build AND know when/what it was generated from.

Marco Pivetta

2 years ago
On Wed, 29 Nov 2023 at 01:48, Matthew Weier O'Phinney < mweierophinney@gmail.com> wrote:
> On Tue, Nov 28, 2023, 5:28 PM Derick Rethans <derick@php.net> wrote: > > > On 28 November 2023 17:28:18 GMT, Sebastian Bergmann <sebastian@php.net> > > wrote: > > > > >While we could probably replace __DATE__ and __TIME__ with > > SOURCE_DATE_EPOCH [3] [4], I cannot help but wonder whether having the > date > > and time when the executable was built in the executable is actually > > useful. How attached are we to having the date and time of the build in > the > > output of phpinfo(), "php -i", etc.? > > > > It is really useful for the development versions of PHP. Knowing whether > > your are running a PHP-dev from last week or last month is important. > > > Would Marco's suggestion of using a git hash solve that? You'd then get > both a reproducible build AND know when/what it was generated from. >
Also, refs have a timestamp :-) Marco Pivetta https://mastodon.social/@ocramius https://ocramius.github.io/

Sebastian Bergmann

2 years ago
Am 29.11.2023 um 01:54 schrieb Marco Pivetta:
> Also, refs have a timestamp :-)
SOURCE_DATE_EPOCH=$(git log -1 --pretty=%cI) should do the trick.

Sebastian Bergmann

2 years ago
Am 29.11.2023 um 07:23 schrieb Sebastian Bergmann:
> SOURCE_DATE_EPOCH=$(git log -1 --pretty=%cI) should do the trick.
What I meant to write was SOURCE_DATE_EPOCH=$(git log -1 --pretty=%ct), of course. Sorry for the noise.

Derick Rethans

2 years ago
On 29 November 2023 00:48:28 GMT, Matthew Weier O'Phinney <mweierophinney@gmail.com> wrote:
>On Tue, Nov 28, 2023, 5:28 PM Derick Rethans <derick@php.net> wrote: > >> On 28 November 2023 17:28:18 GMT, Sebastian Bergmann <sebastian@php.net> >> wrote: >> >> >While we could probably replace __DATE__ and __TIME__ with >> SOURCE_DATE_EPOCH [3] [4], I cannot help but wonder whether having the date >> and time when the executable was built in the executable is actually >> useful. How attached are we to having the date and time of the build in the >> output of phpinfo(), "php -i", etc.? >> >> It is really useful for the development versions of PHP. Knowing whether >> your are running a PHP-dev from last week or last month is important. > > >Would Marco's suggestion of using a git hash solve that? You'd then get >both a reproducible build AND know when/what it was generated from. > >> >> >> >>
Not really, as a hash doesn't directly tell me the date/time, and neither would it help in dev branches / checkouts where the latest changes haven't been comiited yet. cheers Derick

Sebastian Bergmann

2 years ago
Am 29.11.2023 um 08:12 schrieb Derick Rethans:
> Not really, as a hash doesn't directly tell me the date/time, and neither would it help in dev branches / checkouts where the latest changes haven't been comiited yet.
I do not see how date/time help with seeing what was compiled.