[RFC] new benchmark use cases

[Jan Balster]

John Arbash Meinel wrote:
> Jan Balster wrote:
>>> Hi,
>>>
> 
> ...
> 
>>> Can you think of more benchmarks or suggest refinements of existing ones?
>>>
>>>
>>> cheers,
>>>    jan
> 
> In working with the memory tuning, I think having a memory consumption
> per benchmark would also be useful. I think 2 numbers are particularly
> useful, the peak memory consumption, and the total area of memory
> consumption (memory * time)
> So a small jump isn't a big deal, but the absolute max memory
> consumption is important. But just hitting 100MB for a second isn't as
> bad as hitting it and staying there for the whole time.

I agree.

> I'm not sure the best way to measure memory consumption, since python
> defaults to not letting you see how much memory things consume.

There is a python memory profiler http://pysizer.8325.org/ , with which we can
figure out how many memory objects consume.

> Right now, I'm using a wrapper script which spawns bzr, and then read
> /proc/pid/status, and parses it to get the memory size information.

That's probably the most realistic method. Maybe we can use an extended version
of ExternalBase.run_bzr and let the caller decide which version he want to use
at runtime. So we won't have to rewrite the benchmark tests. OTOH every timed
call would include the startup time, which may not be desirable.

> I don't know any cross platform way to get memory consumption, but maybe
> we could factor it out into a function, and add the functionality once
> figure it out.

Right.

> John
> =:->

cheers,
 Jan