[rfc] updated performance-history spec

[Robert Collins]

On Thu, 2006-08-03 at 16:43 +1000, Martin Pool wrote:
> (For context, some performance history is currently displayed here:
> 
> I've added some more notes to the spec
> http://bazaar-vcs.org/PerformanceHistory with a view to making the
> display of history data more useful.  At the moment in 
> http://codespeak.net/bzr/perf_history/summary_bzr.dev.html
> you can see the overall trend for particular benchmarks but it's not
> easy to see what caused them to get better or worse.
> 
> new spec text is
> ----
...
> ----
> 
> Any comments?  Does anyone else think this would be more useful?  If
> so, maybe Holger could implement it?

I think its less useful

I would rather we did one of the earlier suggestions I made which is to
highlight performance drops in the current layout and link from them to
specific revision data - i.e. subordinate pages, or a different section,
or even to bzr webserve.

There are a number of things that combine for me to say its less useful.

The first thing is that the data that matters to me is less accessible:
rather than scanning a few graphs quickly, top to bottom, to see whats
happened within the displayed window, I'll need to scan each change
individually. 

Secondly, totalling the benchmark times across a revision gives a
meaning-free number. It could easily be a 'good value' while we still
have increases in all the user orientated benchmarks, simply because
there is no weighting [possible at all IMO] to combine the benchmarks
from different layers to single 'good/bad' value.

Thirdly, the commit history is not all the interesting to me, as a user
of the benchmarks. Rather starting with the graphs, ordered by
worst-degraded to best-improved, and from there jumping to a revision is
much more useful.

Fourth the mainline revision numbers are kindof ok but make it hard to
use meaningully in different branches. So I'm -1 on that and +1 on using
revision ids.

I think increasing the focus on code changes is best served by starting
with the performance statistics and from there allowing drill-in.

What motivates this changed approach? What do you find lacking in the
current approach, or would like to achieve that you cannot?

Rob


-- 
GPG key available at: <http://www.robertcollins.net/keys.txt>.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 191 bytes
Desc: This is a digitally signed message part
Url : https://lists.ubuntu.com/archives/bazaar/attachments/20060803/eb344f5e/attachment.pgp