[rfc] updated performance-history spec

Martin Pool mbp at canonical.com
Thu Aug 3 07:43:06 BST 2006


(For context, some performance history is currently displayed here:

I've added some more notes to the spec
http://bazaar-vcs.org/PerformanceHistory with a view to making the
display of history data more useful.  At the moment in 
http://codespeak.net/bzr/perf_history/summary_bzr.dev.html
you can see the overall trend for particular benchmarks but it's not
easy to see what caused them to get better or worse.

new spec text is
----

As discussed above, there will be a command to produce an HTML report
from historical data.  This report is intended to answer two main
questions:

* Is performance getting better or worse or staying the same over time?

* What particular changes have caused performance to change?

The first can be answered well by a value-over-time graph of performance
on particular benchmarks and on the whole suite.

The second requires performance numbers to be related to mainline
revisions.  I'd like the page to display each revision number that was
tested, with information like this:


{{{
  revision: 1234
  author: John Hacker <hacker at canonical.com>
  date: 2006-04-01 12:23
  message: merge alleged performance improvements

  results: 404s (+13)
  benchmarks:
    local commit        12.2s (+1)
    status, large tree  
}}}

That is, for each commit we show the current performance, and the delta
relative to the previous commit, for the overall result, and for each
benchmark.  It'd be good if these were highlighted in red or green as
appropriate.  So then we can scan through the page for performance
regressions sticking out in bright red.

It might also be good to produce a similar report from one release to
the next.

Compared to the
[http://codespeak.net/bzr/perf_history/summary_bzr.dev.html current
output] (as of 2006-08-03) this has much more focus on the history of
code changes.

This only has to show the 100 commits, or perhaps even less.  They
should be shown most-recent-first.

----

Any comments?  Does anyone else think this would be more useful?  If
so, maybe Holger could implement it?


-- 
Martin




More information about the bazaar mailing list