Measuring success/failure in the installation

[Scott Kitterman]

On Tuesday, May 17, 2011 06:44:26 AM Evan Dandrea wrote:
> While we have a set of unit tests and a continuous integration system doing
> system testing of the installer, we ultimately have no idea what the
> failure rate is in the real world.
> 
> Without this information, we have no means of actually measuring the true
> quality of the installation experience.  It may look gorgeous and be
> dead-simple to use, but that's worthless if it's not leading to people
> using Ubuntu.
> 
> I am therefore proposing that we actually measure this.  In Oneiric, I
> would like to add code to ubiquity that, once connected to the Internet,
> sends a GUID as generated by uuidgen.  At the end of installation it would
> send this again, and the pair of values in a database would constitute a
> successful installation.  Finally, it will send this value one last time,
> at first boot, to ensure that the system actually works.  From this point
> the GUID will be discarded and never used again.
> 
> I will obviously make the code for this open source, and publish the
> results to a public-facing website.
> 
> The user will be able to disable this functionality by preseeding a
> well-documented key. The documentation will include a brief visual overview
> of how to accomplish this, for those unaccustomed to preseeding the
> installer.
> 
> This addition to the installer will keep us honest. With real data to hand,
> it will be very difficult to ignore the problem if ubiquity regresses in
> its failure rate from release to release.

You knew you'd get this response eventually ...

While I believe you are trying to solve an important problem, I don't think 
this is the right way to go about it.  I do not think a design the phones home 
by default is appropriate for Ubuntu.  I don't think that opt-out via pre-
seeding is acceptable even if on by default is determined to be acceptable as 
it limits this to expert users.  Privacy should be for everyone.

I would recommend instead one more checkbox in the existing setup that invites 
the user to provide installation success information to help improve Ubuntu 
and it should be unchecked by default.

This is something that is a significant change that should be reviewed and 
approved by the Tech Board and possibly the CC.

Scott K