the need for a bench suite

[John Arbash Meinel]

Olivier Grisel wrote:
> Denys Duchier a écrit :
> 
>> I have to say that my modest contributions in trying to speed up bzr
>> have me
>> worried.  I am concerned that by improving one case, I may be introducing
>> regressions elsewhere.
>>
>> It would be really helpful to have a bench suite to help verify that
>> improvements really are, and that they don't introduce regressions.
> 
> 
> +1
> 
>> Probably such a bench suite could be built around unittest, but I
>> don't feel
>> competent to set it up.  If someone could set up just the bare framework
>> (something similar to bzrlib/tests, e.g. bzrlib/bench) and a command
>> to run it
>> (bzr bench), then I would be happy to contribute test cases.
> 
> 
> You might want to reuse existing approaches based on pystones (fore
> machine independence) such as the one that was once in the Zope3 testing
> tools:
> 
> http://svn.zope.org/Zope3/trunk/src/zope/app/testing/performancetest.txt?rev=39009&view=markup
> 
> http://svn.zope.org/Zope3/trunk/src/zope/app/testing/performancetest.py?rev=39009&view=markup
> 
> 
> Regards,
> 

Is there a reason we need a benchmark, rather than just doing:
bzr selftest --verbose
It prints the time taken for each test, and the total time at the end.

Though for real benchmarking, we would want a much bigger test, and I'm
concerned that benchmarking on big trees means we need to carry around a
big tree in the source. How do you test 10k revisions, and 20k files?

Maybe it would be better to have this as a separate branch.

John
=:->


-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 256 bytes
Desc: OpenPGP digital signature
Url : https://lists.ubuntu.com/archives/bazaar/attachments/20060115/6880a19e/attachment.pgp