Everything

php

Replaced the bundled libgd with upstream (aka. system) libgd

php.internals

Christoph Becker

8 years ago

Hi everybody!

During the discussion of PR 3080[1] the idea to replace our bundled
libgd with an (unmodified) upstream libgd[2] has come up again.  It
seems this issue deserves its own discussion, and probably an RFC.

Our bundled libgd (particularly as found in master) is mostly compatible
with upstream libgd 2.2.5 – actually, the API is compatible, but there
are several (mostly minor) implementation differences.  For instance,
the $threshold parameter of imagecropauto($im, IMG_CROP_THRESHOLD) is
interpreted differently[3], the result of imagettfbbox() is slightly
different[4] and imagetruecolortopalette() uses a completely different
algorithm if built with libimagequant (which is not available for the
bundled libgd)[5].  Another important difference is that our bundled
libgd uses ZendMM, but upstream libgd does not[6].

For most Linux environments PHP is built with an upstream (system)
libgd; on Windows usually the bundled libgd is used.  Users targeting
both platforms likely already have conditional code (using `GD_BUNDLED`)
to cater to relevant implementation differences.  So, assuming we would
not set `GD_BUNDLED` anymore after bundling upstream libgd, most users
would not notice a difference.  However, the PHP manual explicitly
recommends to use the bundled libgd[7], so users heeding this advice
would face several minor BC breaks.

The main benefit of bundling upstream libgd, though, would be to save
time by avoiding the 2-way synchronizations (which often occur on a per
commit basis, and may even sometimes be overlooked).

Any thoughts are welcome!

[1] <https://github.com/php/php-src/pull/3080>
[2] <http://libgd.github.io/>
[3] <https://bugs.php.net/73291>
[4] <https://github.com/php/php-src/pull/1845>
[5]
<http://libgd.github.io/manuals/2.2.5/files/gd-h.html#gdPaletteQuantizationMethod>
[6] <https://github.com/libgd/libgd/issues/335>
[7] <http://www.php.net/manual/en/image.installation.php>

--
Christoph M. Becker

Stas Malyshev

8 years ago

Hi!

> During the discussion of PR 3080[1] the idea to replace our bundled
> libgd with an (unmodified) upstream libgd[2] has come up again.  It
> seems this issue deserves its own discussion, and probably an RFC.

I think this is a good idea. In general, as open source ecosystem
matures (and more and more people move to use established OSS vendors or
non-OSS vendors well-integrated into OSS system, like macOS/homebrew
setups) most people move to use distributions which supply wide array of
libraries usually covering ones that we used to bundle, and often doing
better job in keeping with updates and security fixes than we can. So in
general I think phasing out bundling, unless absolutely necessary, is
the idea whose time has come.

> bundled libgd)[5].  Another important difference is that our bundled
> libgd uses ZendMM, but upstream libgd does not[6].

This one we need to find a solution for. GD is often exposed to the
unfiltered user input, has a potential to consume large amounts of
memory and not having ZendMM memory limits in place can be a serious issue.

> For most Linux environments PHP is built with an upstream (system)
> libgd; on Windows usually the bundled libgd is used.  Users targeting

Windows is another concern - are there viable solutions for non-bundled
GD for Windows that we can recommend to the users? If not, that means we
still have to keep and maintain bundled GD, and if so, there's no point
to spend any time on un-bundling before we find solution to this.

--
Stas Malyshev
smalyshev@gmail.com

Christoph Becker

8 years ago

On 07.02.2018 at 21:04, Stanislav Malyshev wrote:

>> bundled libgd)[5].  Another important difference is that our bundled
>> libgd uses ZendMM, but upstream libgd does not[6].
>
> This one we need to find a solution for. GD is often exposed to the
> unfiltered user input, has a potential to consume large amounts of
> memory and not having ZendMM memory limits in place can be a serious issue.

I fully agree.  Presumably few users are aware of this, and even if they
were, it's not easy to cater to this generally.  Until upstream libgd
adds an API to attach custom memory allocators, we might have to patch
the bundled copy (at least this would be a single, well located
modification, instead of the current mess).

>> For most Linux environments PHP is built with an upstream (system)
>> libgd; on Windows usually the bundled libgd is used.  Users targeting
>
> Windows is another concern - are there viable solutions for non-bundled
> GD for Windows that we can recommend to the users? If not, that means we
> still have to keep and maintain bundled GD, and if so, there's no point
> to spend any time on un-bundling before we find solution to this.

I don't see a real issue here.  All other external PHP dependencies are
provided by windows.php.net (amongst others, all GD dependencies, such
as libjpeg), so a libgd.dll could also be provided, especially since
upstream already provides a native Windows build "toolchain" which
already relies on the dependencies provided by windows.php.net (see
<https://github.com/libgd/libgd/tree/gd-2.2.5/windows>).

--
Christoph M. Becker

Anatoliy Belsky

8 years ago

Hi,

> -----Original Message-----
> From: Christoph M. Becker [mailto:cmbecker69@gmx.de]
> Sent: Wednesday, February 7, 2018 11:51 PM
> To: Stanislav Malyshev <smalyshev@gmail.com>; PHP Internals
> <internals@lists.php.net>
> Subject: Re: [PHP-DEV] Replaced the bundled libgd with upstream (aka.
> system)libgd
>
> On 07.02.2018 at 21:04, Stanislav Malyshev wrote:
>
> >> bundled libgd)[5].  Another important difference is that our bundled
> >> libgd uses ZendMM, but upstream libgd does not[6].
> >
> > This one we need to find a solution for. GD is often exposed to the
> > unfiltered user input, has a potential to consume large amounts of
> > memory and not having ZendMM memory limits in place can be a serious
> issue.
>
> I fully agree.  Presumably few users are aware of this, and even if they were, it's
> not easy to cater to this generally.  Until upstream libgd adds an API to attach
> custom memory allocators, we might have to patch the bundled copy (at least
> this would be a single, well located modification, instead of the current mess).
>

There's gdIOCtx in libgd, which allows usage of the PHP streams. Until similar is available for the memory management, we probably stick with a bundled lib in one or another form. Especially on 64-bit.

> >> For most Linux environments PHP is built with an upstream (system)
> >> libgd; on Windows usually the bundled libgd is used.  Users targeting
> >
> > Windows is another concern - are there viable solutions for
> > non-bundled GD for Windows that we can recommend to the users? If not,
> > that means we still have to keep and maintain bundled GD, and if so,
> > there's no point to spend any time on un-bundling before we find solution to
> this.
>
> I don't see a real issue here.  All other external PHP dependencies are provided
> by windows.php.net (amongst others, all GD dependencies, such as libjpeg), so a
> libgd.dll could also be provided, especially since upstream already provides a
> native Windows build "toolchain" which already relies on the dependencies
> provided by windows.php.net (see <https://github.com/libgd/libgd/tree/gd-
> 2.2.5/windows>).
>

Certainly an external libgd is doable. Once the PHP compatibility is there, libgd could be added to the dependency lists. Same scenario as it was done for libzip in 7.2.

From the current POV, it seems that bundling of the patched upstream version, that supports ZMM, would be the optimal solution for both maintenance and security. The unbundling topic can be revived, once libgd provides custom allocator functionality.

Regards

Anatol