[MERGE] more startup benchmarks

[John Arbash Meinel]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Martin Pool wrote:
> On  6 Oct 2006, John Arbash Meinel <john at arbash-meinel.com> wrote:
>> The attached patch adds several more benchmarks which just test the time
>> for simple commands to run.
>>
>> Mostly, I'm trying to add benchmarks which show improvements in startup
>> time. So all of these are meant to be small actions, which will help
>> show that bzr scales down (a simple operation takes a very short amount
>> of time to complete).
>>
>> I'd like to get these merged soon, since it would be nice to see how the
>> startup time decreases as more of my lazy import patches land.
> 
> +1, except that I wonder if we should actually run the commands several
> times (say 10), to make the runtime more measurable?

- From what I've seen, they are all >100ms, which should be reasonably
reproducible. And the biggest fluctuation is when you get a weird
hard-drive blip, and they jump from 200ms to 900ms. Which would still
not average out well when measuring across 5 runs (the extra 700ms still
skews the resuls by 100ms).  It is probably better to just run the
benchmark suite multiple times, and remove outliers/do averaging at that
point. As we discussed for the benchmark changes.

John
=:->

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.5 (Darwin)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFFKvOJJdeBCYSNAAMRApqPAJ9CFxITb7Ss6TdZFt3Zgph3yxR0cACgqQpW
iHlqqXXL5JakvZjaUJAzoyw=
=6UOv
-----END PGP SIGNATURE-----