Everything
php
Learn Community Releases The PHP Project
Home » Project » Newsgroups » php.internals » PHP Benchmark »

PHP Benchmark

php.internals

Sebastian Bergmann

22 years ago
A while ago I posted [1] a script that performed some micro-benchmarks on PHP. I have now refined this benchmark by splitting the monolithic script into one script per micro-benchmark thus performing each benchmark with a fresh PHP interpreter. The benchmark "suite" is available at http://www.sebastian-bergmann.de/PHP_Benchmark/ Any comments, feedback, corrections and additions are welcome, Sebastian
-- [1] http://www.sebastian-bergmann.de/blog/archives/104_PHP_MicroBenchmarkSuite.html -- Sebastian Bergmann http://sebastian-bergmann.de/ http://phpOpenTracker.de/ Das Buch zu PHP 5: http://professionelle-softwareentwicklung-mit-php5.de/

Christian Schneider

22 years ago
Sebastian Bergmann wrote:
> http://www.sebastian-bergmann.de/PHP_Benchmark/ > Any comments, feedback, corrections and additions are welcome,
You've done a good job but we basically only see two things: a) PHP4 is an order of a magniture faster than PHP3. b) PHP5 is about the same speed as PHP4. No big surprises there, let's move on... Regards, - Chris

Mike Robinson

22 years ago
Christian Schneider wrote:
> Sebastian Bergmann wrote: > > http://www.sebastian-bergmann.de/PHP_Benchmark/ > > Any comments, feedback, corrections and additions are welcome, > > You've done a good job but we basically only see two things: > a) PHP4 is an order of a magniture faster than PHP3. > b) PHP5 is about the same speed as PHP4. > > No big surprises there, let's move on...
I think maybe you're missing the point? The benchmark provides insight into changes in PHP speed between versions, major and minor. IMHO that's useful information, nice little sanity check on PHP performance. Just because a change is subtle doesn't mean it's not important. This would be a helpful PEAR addition. Best Regards Mike Robinson

Christian Schneider

22 years ago
Mike Robinson wrote:
> The benchmark provides insight into changes in PHP speed between > versions, major and minor. IMHO that's useful information, nice
It tackles very little of PHP. One of the main changes of PHP5 (object references vs. copying) is missing for example. Of course it could be extended to contain more tests. But you'd also have to make sure that the test environment is always the same (that's why it's nonsense to post individual benchmark results here). Running the benchmark twice will give differing results, even if my quick test showed only around 0.35% difference on a completely idle machine, it's still there. Having a PEAR framework to make benchmarking for core developers easy is A Good Thing(TM). Overestimating the importance of the current benchmark numbers is missing the point. - Chris

George Schlossnagle

22 years ago
On Nov 16, 2003, at 7:44 PM, Christian Schneider wrote:
> Mike Robinson wrote: >> The benchmark provides insight into changes in PHP speed between >> versions, major and minor. IMHO that's useful information, nice > > It tackles very little of PHP. One of the main changes of PHP5 (object > references vs. copying) is missing for example.
You can always add that. It doesn't at all invalidate the results. Thats why they are broken into individual tests instead of being a single monolithic benchmark (which would in fact tell you almost nothing).
> Of course it could be extended to contain more tests. But you'd also > have to make sure that the test environment is always the same (that's > why it's nonsense to post individual benchmark results here).
Welcome to the wonderful world of statistics. I agree posting individual benchmarks is useless (though I can understand the appeal - sorta like posting mips numbers).
> Running the benchmark twice will give differing results, even if my > quick test showed only around 0.35% difference on a completely idle > machine, it's still there.
Welcome to the wonderful world of statistics. Thats why you ensure that you create a consistent environment and use a large enough sample space that your number are statistically significant. George

George Schlossnagle

22 years ago
On Nov 16, 2003, at 5:57 PM, Christian Schneider wrote:
> Sebastian Bergmann wrote: >> http://www.sebastian-bergmann.de/PHP_Benchmark/ >> Any comments, feedback, corrections and additions are welcome, > > You've done a good job but we basically only see two things: > a) PHP4 is an order of a magniture faster than PHP3. > b) PHP5 is about the same speed as PHP4. > > No big surprises there, let's move on...
I think you miss the point. Comprehesive benchmarks are nice so that you can evaluate the effects of and prevent performance accidents often associated withnew changes. It's not groundbreaking stuff, but it's very beneficial to have and to run often. George

Christian Schneider

22 years ago
George Schlossnagle wrote:
> I think you miss the point. Comprehesive benchmarks are nice so that > you can evaluate the effects of and prevent performance accidents often > associated withnew changes. It's not groundbreaking stuff, but it's
I'm just not convinced that these little microbenchmarks are very helpful. To achieve your goal I suggest someone builds benchmarking into the regression tests, i.e. 'make benchmark-tests' or the like, outputting a machine readable format. It would then be possible to automatically compare two versions on a much wider range, allow graphing, alerts if something is much slower etc. Now _that_ sounds interesting for QA. - Chris

George Schlossnagle

22 years ago
On Nov 16, 2003, at 7:53 PM, Christian Schneider wrote:
> > To achieve your goal I suggest someone builds benchmarking into the > regression tests, i.e. 'make benchmark-tests' or the like, outputting > a machine readable format. It would then be possible to automatically > compare two versions on a much wider range, allow graphing, alerts if > something is much slower etc. Now _that_ sounds interesting for QA.
That sounds like good feedback to me. :) George