Initial benchmarks for checkout performance improvements

[Matthew D. Fuller]

On Wed, May 30, 2007 at 02:42:25PM -0400 I heard the voice of
Aaron Bentley, and lo! it spake thus:
> 
> Thanks.  I didn't know about ministat, but it looks quite nice.
> It's good to have Max, Median, and Avg.

Yah, it's handy.  Very simple and limited (only compares 2 data sets,
etc), but great within its area.

What's really nice is its calculation of the difference proven and
error bands at 95% confidence (or sometimes that there's _no_
difference proven 95%).  In this case it doesn't much matter, since
it's glaringly obvious that the difference is "huge" and we don't care
about more precision than that.  But when comparing things that are
closer or with larger deviations and overlap, it beats the pants off
trying to eyeball it, or just comparing means/medians.

Having the stddev printed out is also nice; we can see that with the
checkout outlier, one standard deviation of the checkout time is more
than twice the longest time for transform   ;) And with the outlier
removed, it's still higher than transform's stddev, but less than
cp's.


I'm mildly curious if John ever managed to get ministat building on
Ubuntu, though enough other stuff has been going on since it came up
that I doubt it's been important enough to waste time on.


-- 
Matthew Fuller     (MF4839)   |  fullermd at over-yonder.net
Systems/Network Administrator |  http://www.over-yonder.net/~fullermd/
           On the Internet, nobody can hear you scream.