[MERGE] more startup benchmarks
John Arbash Meinel
john at arbash-meinel.com
Tue Oct 10 02:12:42 BST 2006
-----BEGIN PGP SIGNED MESSAGE-----
Martin Pool wrote:
> On 6 Oct 2006, John Arbash Meinel <john at arbash-meinel.com> wrote:
>> The attached patch adds several more benchmarks which just test the time
>> for simple commands to run.
>> Mostly, I'm trying to add benchmarks which show improvements in startup
>> time. So all of these are meant to be small actions, which will help
>> show that bzr scales down (a simple operation takes a very short amount
>> of time to complete).
>> I'd like to get these merged soon, since it would be nice to see how the
>> startup time decreases as more of my lazy import patches land.
> +1, except that I wonder if we should actually run the commands several
> times (say 10), to make the runtime more measurable?
- From what I've seen, they are all >100ms, which should be reasonably
reproducible. And the biggest fluctuation is when you get a weird
hard-drive blip, and they jump from 200ms to 900ms. Which would still
not average out well when measuring across 5 runs (the extra 700ms still
skews the resuls by 100ms). It is probably better to just run the
benchmark suite multiple times, and remove outliers/do averaging at that
point. As we discussed for the benchmark changes.
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.5 (Darwin)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org
-----END PGP SIGNATURE-----
More information about the bazaar