Measuring success/failure in the installation

[Matt Zimmerman]

On Thu, May 26, 2011 at 07:20:18PM -0700, Kees Cook wrote:
> On Thu, May 26, 2011 at 03:52:45PM +0100, Matt Zimmerman wrote:
> > On Wed, May 25, 2011 at 08:20:30AM -0700, Scott James Remnant wrote:
> > > On Wed, May 18, 2011 at 2:33 AM, Evan Dandrea <ev at ubuntu.com> wrote:
> > > 
> > > > To be clear, since it wasn't addressed in my original email, I intend
> > > > to only present percentages of successful and unsuccessful installs.
> > >  3) Collaboration.
> > > 
> > >     This, to my mind, is the most important.
> > 
> > This is a good reason to share raw data.  It's useful for people to be able
> > to run their own analyses.
> > 
> > That said, the question of whether we share the raw data isn't a
> > deciding factor for me.  I think we should do this measurement because it's
> > useful in itself.
> 
> Can you describe how it is useful? I still don't see why the numbers would
> actually tell us anything actionable. Having a raw count of number of
> installs might be interesting, but I'm not sure I see the step to "useful".

Evan's goal is to measure the effectiveness of the Ubuntu installer by
comparing attempted installations to successful installations.  This will
tell us how many installations fail or are canceled, vs. how many succeed.
Having this data, and being able to measure it over time, will give us
feedback on the changes we make (e.g. did Natty get better or worse compared
to Maverick?).

-- 
 - mdz