Default Desktop Experience for 11.04 - User testing results

[Matthew Paul Thomas]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Bryce Harrington wrote on 15/04/11 06:48:
>
> On Fri, Apr 15, 2011 at 03:00:31AM +0100, Matthew Paul Thomas wrote:
>>
>> *   8/10 people could find a window's menus, but 7/8 of them learned to
>> *   Only 4/11 worked out how to change the background picture.
>> *   6/10 could easily find and launch a game that wasn't in the
>> *   Only 1/9 (P4) easily added that game to the launcher.
>> *   9/11 people could easily close a window.
>> *   8/9 easily copied text from one document into another.
>> *   Only 5/10 could easily delete a document
> 
> These seven items in particular seem like really basic tasks that ought
> to be testing at >90%, so these stats seem a lot lower than I'd expect.
> 
> I'm curious whether these stats are higher/lower/same-as with Classic
> Desktop.  IOW this needs a control group so we can tell if the new
> design brings improvement or regression.

Sure, that's the main problem with using this data. The test was
intended to find fixable problems in Unity, not to compare Unity with
Classic. And the tests Canonical has done on Ubuntu previously have
mostly been on other tasks, such as importing photos and music and
burning CDs.

> Also, these tests measure usability, but not their overall impression.
> Did they like it?  Find it frustrating/confusing?
>...

In my limited experience, user test participants can often be highly
complimentary about an interface that completely failed them (or
critical of an interface where they had no trouble). They can also start
guessing about what other people what other people might think, or
guessing about how easily they might learn something later.

That said, here's a representative sample.

P1: "I've got a Mac, and I think it's quite similar to a Mac, the
layout, which made it easier to use. I think people going from Windows
to this would need a little help, because it's a very different layout."
And: "It's a nice casual-ish font that you've used here."

P2: "I'm guessing the way it's set at the moment, with the wallpaper,
the background, I'd personally want everything to be crisper and clearer
and more obvious ... I find it all a bit blending into each other, with
those settings. But I do like the overall design of it, with the nice
curves and the nice icons at the side, and the font used, and the design
of it's really nice."

P3: "It's prettier than a lot of current operating systems." And: "I
don't think it's a complicated operating system ... it would just take
time."

P4: "Some of the features, like when you minimize something you don't
find the menu bar, so it's quite hard to find." And: "The simple is
quite simple, not too hard to follow or too hard to understand. If
possible, if you could give them a demo. This is where you go to get a
file, this is where you get all your icons, for the first time user,
obviously."

P5: "I really quite like it. I think it's quite intuitive, with the
exception of the favorites, making an application a favorite, which
obviously I wasn't able to achieve. I wouldn't be baffled about how to
use this operating system for the first time, if I didn't have a manual
to read ... It's quite clean-looking, it's quite modern-looking. It
seems to me to be closer to a Mac-style operating system than to a
Microsoft-style operating system."

P7: "No. I don't know. So at this point, I would have to say my initial
impressions are -- I wouldn't write it off, because I've heard too many
good things about it, but my initial impressions are 'Damn, I'm going to
need a manual' ... I don't really want to do that, which is a bit lazy
of me, I know." At the end: "It's not really working for me. I'm finding
it time-consuming and slightly confusing at times ... That's my kind of
bottom-line impression, really, that the differential [between Windows
and Ubuntu], the gap is really very big."

P8: "Maybe the settings should have been more prominent, but I found
them eventually. Fairly standard. If it can run anything I wanted, I
would consider it."

P9: "It's okay ... It's not as confusing as the Mac."

P10: "So generally I think it's pretty good. I think it needs some work
on it. I think making it a bit more intuitive."

P11: "Oh yikes, is it like Apple? ... I never liked the [Mac] interface.
I didn't know where I stored things, where I put things. I like these
pictures. It's aesthetically pleasing. It also looks like it's
child-friendly, because it's pictures. And it looks more ordered than
Apple. So I want to play with it now." At the end: "I think it's pretty.
It's aesthetically pleasing."

P12: "And I really like the cleanness of it all. It's tidy." At the end:
"It's quick and responsive. I'm still frustrated that I'm not able to
find anything like My Computer. And hardware as well, I want to see what
the hardware is, the versions of the graphics and drivers."

- -- 
mpt
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.10 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAk2oOxcACgkQ6PUxNfU6ecrVygCePnJD9A3MyYKjvKWGdaLthYTk
Jv0AnjXtNq1EoHsNXUzG3evF5S4SnCAb
=y15x
-----END PGP SIGNATURE-----