Default Desktop Experience for 11.04 - User testing results

[Rick Spencer]

On Fri, 2011-04-15 at 23:29 +0100, Matthew Paul Thomas wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
> 
> Rick Spencer wrote on 15/04/11 15:08:
> >
> > First, thanks Usability Team! I know how much work goes into planning
> > and running a study like this, and how much agony is involved in
> > interpreting and writing up the results. It's clear that there are some
> > areas for improvement in 11.10, and these results will be instrumental
> > in helping to guide those investments.
> 
> Charline did all the planning and test moderation. I was just the
> stenographer afterwards.
> 
> Later on, Charline will publish a full report on the test. I just wanted
> to post a quick summary in time to be helpful for the default experience
> discussion.
> 
> > On Thu, 2011-04-14 at 22:48 -0700, Bryce Harrington wrote:
> >>
> >> On Fri, Apr 15, 2011 at 03:00:31AM +0100, Matthew Paul Thomas wrote:
> >>>
> >>> *   8/10 people could find a window's menus, but 7/8 of them learned to
> >>> *   Only 4/11 worked out how to change the background picture.
> >>> *   6/10 could easily find and launch a game that wasn't in the
> >>> *   Only 1/9 (P4) easily added that game to the launcher.
> >>> *   9/11 people could easily close a window.
> >>> *   8/9 easily copied text from one document into another.
> >>> *   Only 5/10 could easily delete a document
> >>
> >> These seven items in particular seem like really basic tasks that
> >> ought to be testing at >90%, so these stats seem a lot lower than I'd
> >> expect.
> 
> > Well, think back to the last time you got a new device. For example, if
> > you have an Android phone. You are probably pretty facile with the
> > interface now, but if someone handed it to you and said "do this task
> > with it" you may have struggled to some basic things, like launching
> > apps. A lot of the fun for users in getting a new devices is learning
> > how to use it.
> 
> I didn't have anything close to that kind of trouble when trying out an
> Android phone. (Though like anyone on a developer mailing list, I'm not
> a representative sample.)
Oh? I recall learning many things on my phone. How to launch apps. How
to use the task manager to switch between running apps. How to enter
text in general and swype in particular. How to pull down the
notification area and how to push it up again, etc... If you were
watching over my shoulder you may have said "Fail Fail Fail" many times.
Perhaps you would have been correct, the usabliity could be better.
However, it's easy to forget the first few times you satisficed through
something compared to the hundreds of times you've done something after
you mastered it.

> 
> >...
> > For brand new users? Some of the tasks aren't relevant.
> 
> Which ones?
I believe the following are tasks indicative of repeated and experienced
usage, not first time usage:
 * Changing the background image and setting in general.
 * Adding a game to the launcher.
 * Rearranging items in the launcher.

And these were some that users had the most trouble with.

Cheers, Rick