brainstorming for UDS-N - Performance

[Thierry Carrez]

Kees Cook wrote:
> On Wed, Sep 29, 2010 at 07:43:49PM +0100, Matthew Paul Thomas wrote:
>> Measurement. Where can I go to see the equivalent of Firefox's
>> <http://arewefastyet.com/> for Ubuntu startup speed? Where's the
>> equivalent graph for Ubiquity? For Unity? For Ubuntu Software Center?
>> How much better or worse is yesterday's Natty nightly compared with
>> Ubuntu 10.10? With Ubuntu 10.04 LTS?
> 
> I'm extremely interested in having the various sub-team come up with
> standard measurements so that when people make changes to performance, we
> can actually see it across all of these well-defined workloads.
> 
> I can think of only one that exists now: the boot-time graphs Scott
> manages.
> 
> Changes to the compiler toolchain, the kernel, etc, all have an impact on
> everyone's workloads, but most teams haven't actually stepped forward and
> said "THIS workload is important to us, here's how to reproduce the
> measurement, and here's where we're tracking the daily changes to that
> measurement."

That's something that has been on my mind for server workloads for quite
a bit. We test non-regression in builds, we have some coverage for
non-regression in features, but we have near-zero coverage for
performance regressions, so we get hit (very late) by bugs about how
slow apache2 now serves pages in N compared to N-1, at a point in the
cycle were analysis and drastic solutions are no longer applicable.

I'd definitely welcome a discussion on those topics, be it
server-specific or not.

-- 
Thierry Carrez
Technical lead, Ubuntu server team || Canonical Ltd.