the need for a bench suite

[Johan Rydberg]

John Arbash Meinel <john at arbash-meinel.com> writes:

> Though for real benchmarking, we would want a much bigger test, and I'm
> concerned that benchmarking on big trees means we need to carry around a
> big tree in the source. How do you test 10k revisions, and 20k files?

I guess you could generate random data in random files, given some
parameters such as "in each revision, in average N files is changed,
where each changed file has X new, Y deleted and Z altered lines."
And then use a random generator with a fixed seed, to make the bench
deterministic.

But yes, we need "real benchmarks" as well.  But synthetic ones can be
good to bench "far from average" usage of bzr.

That was my idea a long-long time ago.  

~j