Everything
php
Learn Community Releases The PHP Project
Home » Project » Newsgroups » php.internals » multi-threaded run-tests.php »

multi-threaded run-tests.php

php.internals

Nuno Lopes

19 years ago
Hi, Recently we have moved http://gcov.php.net to a new (and much faster) server. This allowed us to reduce the time to build all branches to "just" 2 days (compared with almost one week that would take previously). However I'm still not happy :P The server has 2x2 CPU cores and I would like to use them :) Currently I can speed-up the make process, because it can be parallelized (with -j xx). But the major time is still spent with run-tests.php. So, what I would like to propose is a multi-threaded version of run-tests.php. We could spawn x processes (configurable) and one of them would test y extensions (I think it's better to have each extension's tests tested serially or they may break, because they may use the same resources: BDs, files, etc..). Any comments? Thanks, Nuno

Richard Lynch

19 years ago
On Tue, July 24, 2007 5:24 pm, Nuno Lopes wrote:
> Recently we have moved http://gcov.php.net to a new (and much faster) > server. This allowed us to reduce the time to build all branches to > "just" 2 > days (compared with almost one week that would take previously). > However I'm still not happy :P The server has 2x2 CPU cores and I > would like > to use them :) Currently I can speed-up the make process, because it > can be > parallelized (with -j xx). But the major time is still spent with > run-tests.php. > > So, what I would like to propose is a multi-threaded version of > run-tests.php. We could spawn x processes (configurable) and one of > them > would test y extensions (I think it's better to have each extension's > tests > tested serially or they may break, because they may use the same > resources: > BDs, files, etc..).
Sounds good to this naive reader. Set X high enough, and you might even find an multi-extension thread-safe issue we've been missing. :-v If I'm reading it right, for sure the pgsql extension tests are serial within the extension, as it re-uses the DB connection and presumes test db setup in early tests, so you're 100% right on the inability to parallelize that.
-- Some people have a "gift" link here. Know what I want? I want you to buy a CD from some indie artist. http://cdbaby.com/browse/from/lynch Yeah, I get a buck. So?

David

19 years ago
Please. This would be a big help to me, since I'm running these tests many, many times a day as I develop the reference cycle collector. On 7/24/07, Richard Lynch <ceo@l-i-e.com> wrote:

Sebastian Bergmann

19 years ago
Nuno Lopes wrote:
> Any comments?
It's on the TODO for the run-tests.php rewrite. I think I/we should start a wiki for this somewhere. Lukas?
-- Sebastian Bergmann http://sebastian-bergmann.de/ GnuPG Key: 0xB85B5D69 / 27A7 2B14 09E4 98CD 6277 0E5B 6867 C514 B85B 5D69

Antony Dovgal

19 years ago
On 25.07.2007 11:33, Sebastian Bergmann wrote:
> Nuno Lopes wrote: >> Any comments? > > It's on the TODO for the run-tests.php rewrite. I think I/we should > start a wiki for this somewhere. Lukas?
Good idea, let's gather all the idea there.
-- Wbr, Antony Dovgal

Nuno Lopes

19 years ago
So here it is my first attempt: http://gcov.php.net/~nlopess/multi_threaded_run_tests.txt WARNING: The patch is ugly and the output of the script isn't the prettiest. Anyway, for automated testing seems to be good enough. Nuno P.S.: after this little hack, I also start to feel that run-tests.php should be rewritten :P ""Nuno Lopes"" <nlopess@php.net> wrote:

Richard Quadling

19 years ago
On 25/07/07, Nuno Lopes <nlopess@php.net> wrote:
> So here it is my first attempt: > http://gcov.php.net/~nlopess/multi_threaded_run_tests.txt > WARNING: The patch is ugly and the output of the script isn't the prettiest. > > Anyway, for automated testing seems to be good enough. > > Nuno > > P.S.: after this little hack, I also start to feel that run-tests.php should > be rewritten :P > > > ""Nuno Lopes"" <nlopess@php.net> wrote: > > Hi, > > > > Recently we have moved http://gcov.php.net to a new (and much faster) > > server. This allowed us to reduce the time to build all branches to "just" > > 2 days (compared with almost one week that would take previously). > > However I'm still not happy :P The server has 2x2 CPU cores and I would > > like to use them :) Currently I can speed-up the make process, because it > > can be parallelized (with -j xx). But the major time is still spent with > > run-tests.php. > > > > So, what I would like to propose is a multi-threaded version of > > run-tests.php. We could spawn x processes (configurable) and one of them > > would test y extensions (I think it's better to have each extension's > > tests tested serially or they may break, because they may use the same > > resources: BDs, files, etc..). > > > > Any comments? > > > > Thanks, > > Nuno
Hey Nuno, you should stop looking at things. PhD and now run-tests. Soon you'll be taking a look at the php-src and thinking, "pah, look what those idiots have done, time for a rewrite!". :-)
-- ----- Richard Quadling Zend Certified Engineer : http://zend.com/zce.php?c=ZEND002498&r=213474731 "Standing on the shoulders of some very clever giants!"

Richard Lynch

19 years ago
On Wed, July 25, 2007 10:39 am, Richard Quadling wrote:
> Hey Nuno, you should stop looking at things. PhD and now run-tests. > Soon you'll be taking a look at the php-src and thinking, "pah, look > what those idiots have done, time for a rewrite!". :-)
Deja-vu: Isn't that exactly what Ze'ev and Andi did for PHP4? :-) Only you have to convince one of your professors that re-writing it instead of doing all your homework is worth a full course credit like they did. :-) Apologies in advance to Ze'ev and Andi for slightly distorting history, I think. (But not much, I hope)
-- Some people have a "gift" link here. Know what I want? I want you to buy a CD from some indie artist. http://cdbaby.com/browse/from/lynch Yeah, I get a buck. So?

Antony Dovgal

19 years ago
On 25.07.2007 19:37, Nuno Lopes wrote:
> So here it is my first attempt: > http://gcov.php.net/~nlopess/multi_threaded_run_tests.txt > WARNING: The patch is ugly and the output of the script isn't the prettiest. > > Anyway, for automated testing seems to be good enough. > > Nuno > > P.S.: after this little hack, I also start to feel that run-tests.php should > be rewritten :P
Unfortunately, this is 100% truth, Sebastian have had some ideas about it.
-- Wbr, Antony Dovgal

Andrey Hristov

19 years ago
Hi Tony, Antony Dovgal wrote:
> On 25.07.2007 19:37, Nuno Lopes wrote: >> So here it is my first attempt: >> http://gcov.php.net/~nlopess/multi_threaded_run_tests.txt >> WARNING: The patch is ugly and the output of the script isn't the >> prettiest. >> >> Anyway, for automated testing seems to be good enough. >> >> Nuno >> >> P.S.: after this little hack, I also start to feel that run-tests.php >> should be rewritten :P > > Unfortunately, this is 100% truth, Sebastian have had some ideas about it. >
I also have a hack, that makes it possible to re-run a test few times. I call it iteration. For every iteration there is $_ENV['TEST_ITERATION']. Based on which different consts are defined by the pre-include script. I am using it for testing mysqlnd against a bunch of MySQL Server, the minimalistic configuration is doing 3 iterations against 4.1, 5.0 and 5.1 Server. Not sure that other extensions could benefit from it, but at least the DB connectivity ones..like testing OCI8.... Regards, Andrey

David Coallier

19 years ago
On 7/25/07, Andrey Hristov <php@hristov.com> wrote:
> Hi Tony, > Antony Dovgal wrote: > > On 25.07.2007 19:37, Nuno Lopes wrote: > >> So here it is my first attempt: > >> http://gcov.php.net/~nlopess/multi_threaded_run_tests.txt > >> WARNING: The patch is ugly and the output of the script isn't the > >> prettiest. > >> > >> Anyway, for automated testing seems to be good enough. > >> > >> Nuno > >> > >> P.S.: after this little hack, I also start to feel that run-tests.php > >> should be rewritten :P > > > > Unfortunately, this is 100% truth, Sebastian have had some ideas about it. > > > > I also have a hack, that makes it possible to re-run a test few times. I > call it iteration. For every iteration there is $_ENV['TEST_ITERATION']. > Based on which different consts are defined by the pre-include script. I > am using it for testing mysqlnd against a bunch of MySQL Server, the > minimalistic configuration is doing 3 iterations against 4.1, 5.0 and > 5.1 Server. Not sure that other extensions could benefit from it, but at > least the DB connectivity ones..like testing OCI8.... >
Where ? :) I am interested in seeing it.
> Regards, > Andrey > > -- > PHP Internals - PHP Runtime Development Mailing List > To unsubscribe, visit: http://www.php.net/unsub.php > >
-- David Coallier, Founder & Software Architect, Agora Production (http://agoraproduction.com) 51.42.06.70.18

Richard Lynch

19 years ago
On Wed, July 25, 2007 12:38 pm, Andrey Hristov wrote:
> Hi Tony, > Antony Dovgal wrote: >> On 25.07.2007 19:37, Nuno Lopes wrote: >>> So here it is my first attempt: >>> http://gcov.php.net/~nlopess/multi_threaded_run_tests.txt >>> WARNING: The patch is ugly and the output of the script isn't the >>> prettiest. >>> >>> Anyway, for automated testing seems to be good enough. >>> >>> Nuno >>> >>> P.S.: after this little hack, I also start to feel that >>> run-tests.php >>> should be rewritten :P >> >> Unfortunately, this is 100% truth, Sebastian have had some ideas >> about it. >> > > I also have a hack, that makes it possible to re-run a test few times. > I > call it iteration. For every iteration there is > $_ENV['TEST_ITERATION']. > Based on which different consts are defined by the pre-include script. > I > am using it for testing mysqlnd against a bunch of MySQL Server, the > minimalistic configuration is doing 3 iterations against 4.1, 5.0 and > 5.1 Server. Not sure that other extensions could benefit from it, but > at > least the DB connectivity ones..like testing OCI8....
FWIW: I spent about 10 to 20 minutes trying to figure out if it was possible to easily run just a subset of tests and failed. I dunno if that indicates that it's a feature to be added, one to be better documented, or that I'm just being stupid (again), but there it is. :-)
-- Some people have a "gift" link here. Know what I want? I want you to buy a CD from some indie artist. http://cdbaby.com/browse/from/lynch Yeah, I get a buck. So?

Rasmus Lerdorf

19 years ago
Richard Lynch wrote:
> On Wed, July 25, 2007 12:38 pm, Andrey Hristov wrote: >> Hi Tony, >> Antony Dovgal wrote: >>> On 25.07.2007 19:37, Nuno Lopes wrote: >>>> So here it is my first attempt: >>>> http://gcov.php.net/~nlopess/multi_threaded_run_tests.txt >>>> WARNING: The patch is ugly and the output of the script isn't the >>>> prettiest. >>>> >>>> Anyway, for automated testing seems to be good enough. >>>> >>>> Nuno >>>> >>>> P.S.: after this little hack, I also start to feel that >>>> run-tests.php >>>> should be rewritten :P >>> Unfortunately, this is 100% truth, Sebastian have had some ideas >>> about it. >>> >> I also have a hack, that makes it possible to re-run a test few times. >> I >> call it iteration. For every iteration there is >> $_ENV['TEST_ITERATION']. >> Based on which different consts are defined by the pre-include script. >> I >> am using it for testing mysqlnd against a bunch of MySQL Server, the >> minimalistic configuration is doing 3 iterations against 4.1, 5.0 and >> 5.1 Server. Not sure that other extensions could benefit from it, but >> at >> least the DB connectivity ones..like testing OCI8.... > > FWIW: > > I spent about 10 to 20 minutes trying to figure out if it was possible > to easily run just a subset of tests and failed. > > I dunno if that indicates that it's a feature to be added, one to be > better documented, or that I'm just being stupid (again), but there it > is. :-)
make test TESTS=ext/foo Or, add this little shell script which I call phpt: #!/bin/bash TEST_PHP_EXECUTABLE=/usr/local/bin/php php /usr/local/lib/php/build/run-tests.php $1 Then cd into whatever directory has your tests and type: phpt . -Rasmus -Rasmus

Sebastian Bergmann

19 years ago
Rasmus Lerdorf wrote:
> make test TESTS=ext/foo > > Or, add this little shell script which I call phpt: > > #!/bin/bash > TEST_PHP_EXECUTABLE=/usr/local/bin/php > php /usr/local/lib/php/build/run-tests.php $1 > > Then cd into whatever directory has your tests and type: > > phpt .
Yeah, letting make take care of this sounds like the right thing to do. It should also solve Ulf's problem of not running the tests for the various MySQL extensions at the same time somehow.
-- Sebastian Bergmann http://sebastian-bergmann.de/ GnuPG Key: 0xB85B5D69 / 27A7 2B14 09E4 98CD 6277 0E5B 6867 C514 B85B 5D69

Marcus Börger

19 years ago
Hello Richard, erm, php run-tests.php -h or --help was to hard? Friday, July 27, 2007, 5:56:10 AM, you wrote:
> On Wed, July 25, 2007 12:38 pm, Andrey Hristov wrote: >> Hi Tony, >> Antony Dovgal wrote: >>> On 25.07.2007 19:37, Nuno Lopes wrote: >>>> So here it is my first attempt: >>>> http://gcov.php.net/~nlopess/multi_threaded_run_tests.txt >>>> WARNING: The patch is ugly and the output of the script isn't the >>>> prettiest. >>>> >>>> Anyway, for automated testing seems to be good enough. >>>> >>>> Nuno >>>> >>>> P.S.: after this little hack, I also start to feel that >>>> run-tests.php >>>> should be rewritten >>> >>> Unfortunately, this is 100% truth, Sebastian have had some ideas >>> about it. >>> >> >> I also have a hack, that makes it possible to re-run a test few times. >> I >> call it iteration. For every iteration there is >> $_ENV['TEST_ITERATION']. >> Based on which different consts are defined by the pre-include script. >> I >> am using it for testing mysqlnd against a bunch of MySQL Server, the >> minimalistic configuration is doing 3 iterations against 4.1, 5.0 and >> 5.1 Server. Not sure that other extensions could benefit from it, but >> at >> least the DB connectivity ones..like testing OCI8....
> FWIW:
> I spent about 10 to 20 minutes trying to figure out if it was possible > to easily run just a subset of tests and failed.
> I dunno if that indicates that it's a feature to be added, one to be > better documented, or that I'm just being stupid (again), but there it > is.
> -- > Some people have a "gift" link here. > Know what I want? > I want you to buy a CD from some indie artist. > http://cdbaby.com/browse/from/lynch > Yeah, I get a buck. So?
Best regards, Marcus