Everything

php

Source tar.gz's extract slowly on some platforms

php.internals

Thomas Hruska

9 years ago

I've been noticing lately that the newer source code tar.gz files are
taking longer to extract on some platforms.  So I decided to watch an
extraction time for some anecdotal evidence.  Using 7-Zip, the 7.1.4
source files took 2 minutes to extract to a SSD drive on my Windows
machine.  The archive itself contains 19,155 files and is ~20MB in size.
I suspect that there are inefficiencies with writing lots of new,
small files to disk.

It looks like ~700 files are added each year to whatever new version is
released.  The bulk of the new files seem to involve the test suite.

I'm only pointing out something I've noticed.  I'm not compiling
anything when I just want to search/view the latest source code nor do I
run 'make test' when deploying, so the test suite is somewhat
superfluous for most of my use-cases.  Test suites are useful and
important, which makes this an observation unless, of course, this
prompts someone to go forth and do something (whatever that might be).

--
Thomas Hruska
CubicleSoft President

I've got great, time saving software that you will find useful.

http://cubiclesoft.com/

Sara Golemon

9 years ago

On Sat, Apr 22, 2017 at 8:38 AM, Thomas Hruska <thruska@cubiclesoft.com> wrote:

> It looks like ~700 files are added each year to whatever new version is
> released.  The bulk of the new files seem to involve the test suite.
>

I certainly hope the vast bulk of that is tests. :D

> I'm only pointing out something I've noticed.  I'm not compiling anything
> when I just want to search/view the latest source code nor do I run 'make
> test' when deploying, so the test suite is somewhat superfluous for most of
> my use-cases.  Test suites are useful and important, which makes this an
> observation unless, of course, this prompts someone to go forth and do
> something (whatever that might be).
>

I hear what you're saying in terms of a "browsing source code" use
case, but given availability of tools like lxr.php.net and of course
github.com/php/php-src the scope of that use case feels quite small
compared to a one-time decompression cost, even at seven minutes.

But if your workflow is your workflow and tarballs are it, consider
using the `--exclude` directive to `tar`.

Example: tar -zxf php-7.1.0.tgz --exclude=tests

-Sara

Rowan Collins

9 years ago

On 22/04/2017 17:30, Sara Golemon wrote:

> availability of tools like lxr.php.net

Sorry, I just have to comment on that phrase; lxr.php.net is an
absolutely awesome tool, but its "availability" is ... well, let's say
it's not exactly had five nines uptime lately. :(

Right now, I get a "502 Proxy Error" just going to the homepage; I think
last time I tried, I could load the search form, but got an error
whenever I tried to search for something.

I'm sure there's good reasons why it needs more than turning off and on
again, but if anyone has the skills and time to volunteer, it would make
me very happy if it were brought back to life. Or, if there is some
other tool out there like it, I'd be very grateful for a pointer.

Regards,

--
Rowan Collins
[IMSoP]

Christoph Becker

9 years ago

On 22.04.2017 at 19:56, Rowan Collins wrote:

> On 22/04/2017 17:30, Sara Golemon wrote:
>
>> availability of tools like lxr.php.net
>
> Sorry, I just have to comment on that phrase; lxr.php.net is an
> absolutely awesome tool, but its "availability" is ... well, let's say
> it's not exactly had five nines uptime lately. :(
>
> Right now, I get a "502 Proxy Error" just going to the homepage; I think
> last time I tried, I could load the search form, but got an error
> whenever I tried to search for something.
>
> I'm sure there's good reasons why it needs more than turning off and on
> again, but if anyone has the skills and time to volunteer, it would make
> me very happy if it were brought back to life. Or, if there is some
> other tool out there like it, I'd be very grateful for a pointer.

Thanks to Adam, there is <https://php-lxr.adamharvey.name/source/>.

--
Christoph M. Becker

Rowan Collins

9 years ago

On 23/04/2017 09:36, Christoph M. Becker wrote:

> Thanks to Adam, there is <https://php-lxr.adamharvey.name/source/>.

Yay! Adam, I love you! :)

And cheers to you, Christoph, for the link!

--
Rowan Collins
[IMSoP]

Chris Wright

9 years ago

On 23 April 2017 at 17:24, Rowan Collins <rowan.collins@gmail.com> wrote:

> On 23/04/2017 09:36, Christoph M. Becker wrote:
>
>> Thanks to Adam, there is <https://php-lxr.adamharvey.name/source/>.
>>
>
> Yay! Adam, I love you! :)
>
> And cheers to you, Christoph, for the link!
>
> --
> Rowan Collins
> [IMSoP]
>
>

We are also running OpenGrok at https://lxr.room11.org/, it has php-src @
5.6, 7.0, 7.1 and master, as well as a selection of other projects. It's
been pretty a pretty stable installation as far as we can tell, although
I'm sure I have now cursed it by making such a foolish assertion in public.

It's a slightly more recent version than php.net and Adam's, with some
shinier UI features and a JSON API (
https://github.com/OpenGrok/OpenGrok/wiki/OpenGrok-web-services).

I have built a docker image for it and it is possible, in theory at least,
to deploy a new instance in half an hour or so. If anyone would like more
info feel free to contact me directly :-)

Dan Ackroyd

9 years ago

On 22 April 2017 at 16:38, Thomas Hruska <thruska@cubiclesoft.com> wrote:

> Using 7-Zip, the 7.1.4 source files took
> 2 minutes to extract to a SSD drive on my Windows machine.

I noticed this also, when testing a reasonably powerful Windows
desktop machine that I had intended to use as my new development +
continual build machine.

I then measured the time taken to extract the source on my seven year
old MacBookPro; it takes seven seconds to extract the file on that
machine.

The conclusion I made was that some OS's just aren't that good a fit
if you're going to be extracting files on a regular basis. ¯\_(ツ)_/¯

cheers
Dan