[RFC] tweak merge-approval criterion: performance

[Ian Clatworthy]

Alexander Belchenko wrote:
> Martin Pool ?8H5B:
>> We do actually have a machine doing that, at
>> http://benchmark.bazaar-vcs.org/  The output at the moment is a bit
>> technical, with the summary table right down the bottom.  It does not
>> send mail at present.
> 
> Newer saw this URL.

It's a good start but it's still pretty crude. We haven't mentioned it
previously because the data requires a fair amount of "interpretation".
In other words, check the results over several runs before reading too
much into the data.

> Report is interesting and show something that looks like regression:
> 
> + date -u
> Tue Aug  7 07:23:31 UTC 2007
> + echo
> 
> --- Measurements of time ---
> Tools:                          bzr        bzr-0.18        bzr-0.17
> time add                     10.566           8.313           7.710 (s)
> time status                  30.466          30.289          10.202 (s)
> 
> 
> And this one is really cool:
> 
> --- Measurements of time ---
> Tools:                          bzr        bzr-0.18        bzr-0.17
> time bundle                   0.207          20.309          20.291 (s)

Hmm. This could be mega impressive. It might also be triggering a bug or
crash. Note that the tool just runs commands and times them, i.e. only
performance, not correctness. It doesn't check whether the commands
actually worked. :-( See point 1 about being pretty crude still.

Ian C.