Measuring success/failure in the installation

Ken VanDine ken.vandine at
Tue May 17 13:07:09 UTC 2011

On Tue, 2011-05-17 at 11:44 +0100, Evan Dandrea wrote:
> While we have a set of unit tests and a continuous integration system doing
> system testing of the installer, we ultimately have no idea what the failure
> rate is in the real world.
> Without this information, we have no means of actually measuring the true
> quality of the installation experience.  It may look gorgeous and be
> dead-simple to use, but that's worthless if it's not leading to people using
> Ubuntu.
> I am therefore proposing that we actually measure this.  In Oneiric, I would
> like to add code to ubiquity that, once connected to the Internet, sends a
> GUID as generated by uuidgen.  At the end of installation it would send this
> again, and the pair of values in a database would constitute a successful
> installation.  Finally, it will send this value one last time, at first boot,
> to ensure that the system actually works.  From this point the GUID will be
> discarded and never used again.
> I will obviously make the code for this open source, and publish the results
> to a public-facing website.
> The user will be able to disable this functionality by preseeding a
> well-documented key. The documentation will include a brief visual overview of
> how to accomplish this, for those unaccustomed to preseeding the installer.
> This addition to the installer will keep us honest. With real data to hand, it
> will be very difficult to ignore the problem if ubiquity regresses in its
> failure rate from release to release.

I am very much in favor of this.  We are trying to reach a wider
audience, and not only do we have no way of measuring our success, we
also have no way of knowing if the users are successfully installing

I think efforts like this are necessary to take the quality of Ubuntu to
the next level.  I understand some people are pretty concerned about how
data like this will be used, however having the code public should
ensure we are doing the right thing.

Thanks for taking the initiative, lets set the quality bar high!

Ken VanDine
Ubuntu Desktop Integration Engineer
Canonical, Ltd.

More information about the ubuntu-devel mailing list