Default Desktop Experience for 11.04 - User testing results

[Rick Spencer]

First, thanks Usability Team! I know how much work goes into planning
and running a study like this, and how much agony is involved in
interpreting and writing up the results. It's clear that there are some
areas for improvement in 11.10, and these results will be instrumental
in helping to guide those investments.

On Thu, 2011-04-14 at 22:48 -0700, Bryce Harrington wrote:
> On Fri, Apr 15, 2011 at 03:00:31AM +0100, Matthew Paul Thomas wrote:
> > *   8/10 people could find a window's menus, but 7/8 of them learned to
> > *   Only 4/11 worked out how to change the background picture.
> > *   6/10 could easily find and launch a game that wasn't in the
> > *   Only 1/9 (P4) easily added that game to the launcher.
> > *   9/11 people could easily close a window.
> > *   8/9 easily copied text from one document into another.
> > *   Only 5/10 could easily delete a document
> 
> These seven items in particular seem like really basic tasks that ought
> to be testing at >90%, so these stats seem a lot lower than I'd expect.
Well, think back to the last time you got a new device. For example, if
you have an Android phone. You are probably pretty facile with the
interface now, but if someone handed it to you and said "do this task
with it" you may have struggled to some basic things, like launching
apps. A lot of the fun for users in getting a new devices is learning
how to use it.

Given my experience, these numbers look they could be improved, but I
don't find them particularly concerning. For example, you can easily use
Unity quite productively before you learn that you can add items to the
launcher, or change the background picture, though I suspect many
Windows users would right-click and the desktop to set the background
picture and would do fine.

The only area here that is at all concerning to me regards launching
applications. I'd like to see some focus on the application lens in
11.10 (see what I did I did there? :) )

> 
> I'm curious whether these stats are higher/lower/same-as with Classic
> Desktop.  IOW this needs a control group so we can tell if the new
> design brings improvement or regression.
For brand new users? Some of the tasks aren't relevant. For others, like
finding installed applications, I presume this was dead easy in Classic
GNOME, but it's hard to say for a certain.

> 
> Also, these tests measure usability, but not their overall impression.
> Did they like it?  Find it frustrating/confusing?
This is actually a very important question. For me, when I go back to
Classic, it feels very old and mundane. Many theories hold that the
aesthetics trump usability, or at least strongly influence the
perception of the usability of a system. In other words, given 2
identically design systems that only differ in terms of theming, for
example, users will rate the system with the more pleasing design to be
more "usable" and are more likely to start using it. 

Cheers, Rick