benchmark selftests

[Martin Pool]

On 22 May 2006, John Arbash Meinel <john at arbash-meinel.com> wrote:
> I like the ideas, I just don't want us to be hamstrung by a commit policy.

I think logging the performance would be good; I wouldn't want to start
rejecting commits based on it certainly for a long time.  

At the moment we have a bit of a target rich environment and the big
thing is to add more benchmarks and fix the obvious things they point
out.

Once we are through that we'll want to watch for any performance
regressions from one release to the next, but not necessarily in every
commit that comes along.  As you say it's fine to temporarily get slower
to clean up the code or fix a bug -- conceivably the previous version was
fast but unsafe, and it's impossible to make correct code be that fast.
Still it could be good to get at least a warning if something suddenly
gets worse.

For the moment if would just be nice to have the system tell you, in any
branch, whether you're making things worse or better.

-- 
Martin