[RFC] Benchmark configuration handling

[Carl Friedrich Bolz]

Robert Collins wrote:
> On Fri, 2006-07-28 at 10:16 +0200, Carl Friedrich Bolz wrote:
>> What's the preferred way to handle configuration for benchmarks? The
>> problem is that the number of options for a benchmark run will likely
>> increase: the times that the SocketDelay sleeps per roundtrip and per
>> transmitted character are the first things, but other options might
>> follow.  (The benchmark config file probably also should contain
>> information how to produce a web page so that people can
>> easily publish results from certain benchmarking settings at
>> some point).
>
> Erm, I think each benchmark test should have its settings fixed, I see
> no reason to configure or change them: benchmark identity would be
> valueless if they are altered. For the same reason we try not to change
> benchmarks after they are accepted into the mainline.

Sure, I agree with all of this. I was fearing a sort of combinatorical
explosion, but it's probably more sensible to choose some particular
configurations that you are interested in anyway and add benchmarks for
those. Are there benchmarks that are particularly interesting to run
using the slow socket?

[snip]

Cheers,

Carl Friedrich