Measuring success/failure in the installation

[Evan Dandrea]

There appears to be consensus on the following ubuntu-devel thread
that this issue requires a decision by the tech board.  May I kindly
ask that you address it at the meeting tomorrow?

https://lists.ubuntu.com/archives/ubuntu-devel/2011-May/033194.html

To be clear, since it wasn't addressed in my original email, I intend
to only present percentages of successful and unsuccessful installs.
I do not want this misused as a (very poor) metric for counting users
when many wont have an Internet connection, will get their Ubuntu
system from an OEM, or will have installed once and upgraded through
each release.

I would also like to single out Steve's response, as it more
eloquently presents some of the arguments I was attempting to make:

https://lists.ubuntu.com/archives/ubuntu-devel/2011-May/033211.html


---------- Forwarded message ----------
From: Evan Dandrea <ev at ubuntu.com>
Date: Tue, May 17, 2011 at 11:44 AM
Subject: Measuring success/failure in the installation
To: ubuntu devel <ubuntu-devel at lists.ubuntu.com>


While we have a set of unit tests and a continuous integration system doing
system testing of the installer, we ultimately have no idea what the failure
rate is in the real world.

Without this information, we have no means of actually measuring the true
quality of the installation experience.  It may look gorgeous and be
dead-simple to use, but that's worthless if it's not leading to people using
Ubuntu.

I am therefore proposing that we actually measure this.  In Oneiric, I would
like to add code to ubiquity that, once connected to the Internet, sends a
GUID as generated by uuidgen.  At the end of installation it would send this
again, and the pair of values in a database would constitute a successful
installation.  Finally, it will send this value one last time, at first boot,
to ensure that the system actually works.  From this point the GUID will be
discarded and never used again.

I will obviously make the code for this open source, and publish the results
to a public-facing website.

The user will be able to disable this functionality by preseeding a
well-documented key. The documentation will include a brief visual overview of
how to accomplish this, for those unaccustomed to preseeding the installer.

This addition to the installer will keep us honest. With real data to hand, it
will be very difficult to ignore the problem if ubiquity regresses in its
failure rate from release to release.