[PATCH] performance history specification

[Robert Collins]

On Fri, 2006-07-14 at 10:04 +0200, Jan Balster wrote:
> 
> Robert Collins wrote:
> > On Thu, 2006-07-13 at 23:21 -0500, John Arbash Meinel wrote:
> ...
> > 
> > I'd like it in asap so that we can put a cron job on bazaar-vcs.org
> > running HEAD with this, so we have object performance stats for all the
> > benchmarks.
> > 
> 
> Hi Rob,
> 
> I created a preliminary web page (and a little cron job on codespeak.net)
> for showing performance numbers from the bzr.dev main branch.
> The page is available at http://codespeak.net/bzr/perf_history/summary_bzr.dev.html
> 
> It's designed to give an overview of the development on the performance front.
> Comments/thoughts? (let's talk about fonts and colors later  ;)  )

I think there are distinct graphs that are useful:
1) overall performance, release to release. One way this could be done
is by averaging the metrics for each sample to get a single figure, then
graphing them all, and marking releases with a vertical mark of some
sort - a bold plot point, or a vertical bar, or whatever. This is
clearly not completely ideal as the different benchmarks are on
different scales - but I figure after a certain point we'll become
'done' as far as introducing benchmarks.

2) Detailed performance of development within this release cycle,
probably the last week is in fact the most interesting thing to be
looking at.

A few other random thoughts:
AddBenchmark.test_one_add_kernel_like_tree - this would be better to
show than the full python path to the test - just grab the last two
segments after splitting on '.'.

Perhaps put the name at the left?

Perhaps sort with the benchmarks that have the most improvement at the
bottom - so the ones with the least improvement, or (shock) benchmarks
that have gotten worse at the top.
Its this sort that having only the last weeks data shown would be most
useful for, because it gives a locality of reference to the analysis.

Possibly highlight rows that have got worse with a different
shade/inverse colours or something.

Rob


-- 
GPG key available at: <http://www.robertcollins.net/keys.txt>.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 191 bytes
Desc: This is a digitally signed message part
Url : https://lists.ubuntu.com/archives/bazaar/attachments/20060714/9af871da/attachment.pgp