Usability Testing [was: Re: FAQ page]

[Matthew Nuzum]

On Fri, 2006-07-28 at 21:03 +0200, Duncan Lithgow wrote:
> On Fri, 2006-07-28 at 11:29 -0500, Matthew Nuzum wrote: 
> > On Fri, 2006-07-28 at 12:00 +0200, Andreas Lloyd wrote:
> > > Duncan Lithgow wrote:
> > > > If someone could make a spec with differently graded tasks, I'd be happy
> > > > to record desktop sessions and get some friends and family to carry out
> > > > the listed tasks. But writing the tasks and instructions should be done
> > > > by someone with good understanding.
> > 
> > I can do this. I'll work on it either today or shortly after. I've got a
> > bit of a backlog...
> It sounds great, maybe you can grade them. Like:
> 
> 'quick' - takes five minutes and a friend
> 'basic' - takes up to 15 minutes, and needs sound recording
> 'detailed' - takes 3 cameras, and sound recording
> 

I've created three specs here:
https://launchpad.net/products/ubuntu-website/+specs

I will more fully develop a process but for now, here are two things you
can do that would probably be good starters:

Get friends who are not familiar with the website to do the following
two tasks: (using a variety of computer OSs preferably, Win 98, XP, Mac,
Linux)
 * Download the relevant Ubuntu CD Image and burn it to a CD. 
 * Order CDs from shipit.

You don't need to record anything, just clearly define the task and tell
the potential tester that you want to see how easy or difficult the task
is. For example,
"There is a cool new product called Ubuntu. You can log into their
website and download the CD image to burn to a blank CD for free. Then
you can install it on a computer. My team is interested in seeing how
easy or difficult it is for people to download and create the CD. Would
you mind trying to download it while I quietly watch? I want to see if
you can do it without having to ask for help, so pretend I'm not here."

Make notes of where people get stuck. If they seem to spend a lot of
time on a particular page or have to bounce back and forth a couple
times, make note of this.

Also make note of the type of computer OS they're using. We may find
that Windows XP users have no troubles but Mac users struggle.

Don't wait to take notes. Do it during and/or immediately after.

There are two different types of observations: Subjective and Objective.
Subjective observations are, "it seems like..." while objective are "it
too 8 clicks..." We are interested in both but please clearly note when
you are expressing an opinion and as much data as possible that lead you
to the opinion. For example, "I noticed that users clicked back to the
XYZ page several times before finding the correct link to PDQ. Therefore
I think we need to make PDQ higher on the page." The subjective part is
that PDQ needs to be higher. The objective part is that people had to
click back to the page several times.

Also note that this terminology of "clicking back to" is kind of vague
and you might want to be more specific.

Feedback and comments are emphatically welcome.
-- 
Matthew Nuzum
newz2000 on freenode