Everything

php

php 5.2.4 test results reference

php.internals

jean jayet

18 years ago

Hi folks

working at Sun Microsystems QA org we are integrating php into our new
Open Solaris release (php 5.2.4 for now)

we have run the functional test suite shipped within the php release to
address/assess non regression testing area

similarly we have run the php 5.2.4 test suite vs a Red Hat 4 platform
to generate a base of comparison for test results between Solaris and RH

as we have caught some failed tests on Solaris (less on RH) (detailed
test report is available on our side)

we would like to know whether at php org level there are any test
results available / published / shared per OS (like Solaris, RH in
particular)

in order we can compare the results obtained on Solaris with some
"official run" and therefore get a status about failed tests we caught :
typically are there known bugs per OS (corresponding to failed tests)
already filed for php 5.2.4 release

thks in advance
Jean

Derick Rethans

18 years ago

On Wed, 17 Oct 2007, jean jayet wrote:

> we would like to know whether at php org level there are any test
> results available / published / shared per OS (like Solaris, RH in
> particular)

Not grouped, but we do have all the test results here:
http://news.php.net/php.qa

regards,
Derick

--
Derick Rethans
http://derickrethans.nl | http://ez.no | http://xdebug.org

jean jayet

18 years ago

Hi Derick

many thks for the pointer

going thru this link, there are hundreds of reports posted !!! can not
go thru all of them

most of the reports opened are for Linux

didn't find any dedicated to Solaris (but for sure assuming they exist)

parsing results, tests failed may vary (even if noticing same failed
tests often occur)

i have the following naive question : how can get a reference of test
results

smth to be able to find :

such set of test case id MUST pass
such set test case id MUST fail
such set test case id skip if such extension missing etc,...

thks
Jean

Derick Rethans wrote On 10/17/07 02:41 PM,:

Derick Rethans

18 years ago

On Wed, 17 Oct 2007, jean jayet wrote:

> i have the following naive question : how can get a reference of test
> results
>
> smth to be able to find :
>
> such set of test case id MUST pass
> such set test case id MUST fail
> such set test case id skip if such extension missing etc,...

Well, they should all pass - skipped tests depends totally on the
configuration - I don't think we've any information on that at all.

regards,
Derick

jean jayet

18 years ago

> Well, they should all pass

as catching 50 failed tests for php 5.2.4 / 5.11

for failed tests can someone tell the action to take ?

- report a new bug at php org level for this php release ?
- integration problem specific to OS (eg implementation specifities) ?
- other ?

rgds,
Jean

Derick Rethans wrote On 10/17/07 03:51 PM,:

Antony Dovgal

18 years ago

On 17.10.2007 18:05, jean jayet wrote:

>> Well, they should all pass
>
> as catching 50 failed tests for php 5.2.4 / 5.11
>
> for failed tests can someone tell the action to take ?

Figure out the reason and act accordingly.
If it's a problem caused by some Solaris-specific behaviour and we're missing a
workaround for it (which is obvious for you) - send us a patch.
If the test is wrong - report it to the php.qa list (or here).
If there is a bug in PHP - report to bugs.php.net.

--
Wbr,
Antony Dovgal

Zoe Slattery

18 years ago

Derick Rethans wrote:

> On Wed, 17 Oct 2007, jean jayet wrote:
>
>
>> i have the following naive question : how can get a reference of test
>> results
>>
>> smth to be able to find :
>>
>> such set of test case id MUST pass
>> such set test case id MUST fail
>> such set test case id skip if such extension missing etc,...
>>
>
> Well, they should all pass - skipped tests depends totally on the
> configuration - I don't think we've any information on that at all.
>
>

Jean - if the tests don't pass on your system it would be helpful if you
could raise defects (http://bugs.php.net/) checking first to make sure
that someone hasn't already done so, and  make the .diff and .out files
available. In some cases when we write tests it isn't possible to test
on ALL platforms, so sometimes the tests may fail because they really
should not pass on a specific platform (if so we will fix by adding a
--SKIPIFF-- to the test case), sometimes it's because of an
implementation difference - we can't tell till we see the output from
the test.

As Derick says, they should all pass :-)

Zoe

Nuno Lopes

18 years ago

>> we would like to know whether at php org level there are any test
>> results available / published / shared per OS (like Solaris, RH in
>> particular)
>
> Not grouped, but we do have all the test results here:
> http://news.php.net/php.qa

In addition to what Derick said, there is a "reference" build and test
failure at http://gcov.php.net

Nuno

Marcus Börger

18 years ago

Hello Nuno,

we should work on that database thing that collects information from
trusted gcov machines.....

marcus

Wednesday, October 17, 2007, 11:03:47 PM, you wrote:

>>> we would like to know whether at php org level there are any test
>>> results available / published / shared per OS (like Solaris, RH in
>>> particular)
>>
>> Not grouped, but we do have all the test results here:
>> http://news.php.net/php.qa

> In addition to what Derick said, there is a "reference" build and test
> failure at http://gcov.php.net

> Nuno

Best regards,
Marcus

Nuno Lopes

18 years ago

I know, I know.. It is on my TODO list for a loong time. I have the program
on my head, I just need to get some time to sit and code it..

BTW: Jean, would Sun be interested in setting up a build machine to run the
PHP test suite and post the results to a central build DB?

Nuno


----- Original Message -----

jean jayet

18 years ago

(resending this message as got return errors)


Thanks Nuno for this proposition

for sure Sun is very interested in publishing/sharing the test results
of integrating PHP 5.2.4 in its OpenSolaris version (Solaris Express
Developer Edition, SXDE)

fyi, the PHP 5.2.4 test suite has already been executed on a platform
with SXDE build : plan is to include other OSS components like Apache,
MySQL, Squid, etc,...to complete the web-base stack called Sun WebStack

same php test suite has been executed on Red Hat 4 to establish a
comparison of tests results (building PHP with the same way of course on
these 2 OS)

as test results have not been yet reported to PHP QA, can we share with
you the results obtained ?

we would like to know whether possible to get your help to analyze the
reasons of the test failures (tests results of PHP 5.2.4 for SXDE OS are
inclosed)

we have been said all tests should pass

before filing any php bug we would like to know if failures are due to
tests issues requesting test updates/patches, integration issues from
us, Solaris specificities, real php bugs for 5.2.4 release etc,...


thanks in advance for your support
Jean



Nuno Lopes wrote On 10/18/07 11:24 PM,: