Measuring success/failure in the installation

[Steve Langasek]

Hi Scott,

On Tue, May 17, 2011 at 09:31:59AM -0400, Scott Kitterman wrote:
> You knew you'd get this response eventually ...

:-)

> While I believe you are trying to solve an important problem, I don't think 
> this is the right way to go about it.  I do not think a design the phones home 
> by default is appropriate for Ubuntu.  I don't think that opt-out via pre-
> seeding is acceptable even if on by default is determined to be acceptable as 
> it limits this to expert users.  Privacy should be for everyone.

I believe that an installed Ubuntu system will also contact
security.ubuntu.com by default (regardless of what mirror one selects for
the main archive).  From the user's perspective, is there any difference in
the information exposed via the GUID tracking method, vs. having each
machine contacting the central security.u.c server?

 - neither method includes identifying personal information
 - both methods can be used to get a rough count of the number of installs
   behind a given IP address, if someone on the Canonical side chose to
   track this (the GUID gives a direct count of new machine installs;
   security.u.c gives an approximate count of active machines based on
   number of apt-get update runs per day)
 - both methods can be screened by use of a firewall / web proxy

I too am concerned that we not violate users' privacy by gathering
identifying information from them without their consent - even if Canonical
has no designs on using this information maliciously, I don't think even the
temptation should be there.  But if the information being gathered here
is equivalent to other information we already have in terms of sensitivity,
while giving the installer team better tools to improve the installer
experience, I would hate to see this initiative be hamstrung by privacy
questions that don't actually benefit the user.

> I would recommend instead one more checkbox in the existing setup that invites 
> the user to provide installation success information to help improve Ubuntu 
> and it should be unchecked by default.

This may bias the incoming data in ways that are not easy to analyze.  Are
users who opt in more likely to have a successful install because they pay
more attention to details as they go?  Less likely, because they are
tweaking things and will hit other breakage due to hitting uncommon paths?
More likely, because there's a correlation between willingness to
participate and willingness to persevere in the face of installer glitches?
Less likely, because experienced users won't want to participate?

The one thing we can say for sure is that if this isn't enabled by default,
we will miss out on capturing this data for the least engaged subset of
users, who in a sense are also the most interesting group to get this
information for:  we can usefully extrapolate from unengaged users as a
lower bound for installation success, we can't do the converse to ascertain
anything about the experience of the unengaged users by looking at the
results from more heavily engaged users.

> This is something that is a significant change that should be reviewed and 
> approved by the Tech Board and possibly the CC.

I think it's a very good idea to put this question to the TB.

-- 
Steve Langasek                   Give me a lever long enough and a Free OS
Debian Developer                   to set it on, and I can move the world.
Ubuntu Developer                                    http://www.debian.org/
slangasek at ubuntu.com                                     vorlon at debian.org
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 828 bytes
Desc: Digital signature
URL: <https://lists.ubuntu.com/archives/ubuntu-devel/attachments/20110517/14d78b60/attachment.pgp>