Usability Testing

[Celeste Lyn Paul]

Chuck forwarded me the notes from last night and we talked a little about your 
idea for usability testing in the LoCo.  I think this is a great idea if we 
plan it right.  I added it as an agenda item for our next IRC meeting, but I 
also wanted to open the conversation up on the mailing list.

What to test:

Your first reaction is probably the installation process, but to tell you the 
truth Ubiquity has been tested to death.  It has had two usability tests 
conducted on it and multiple UI reviews (I just submitted some UI bugs the 
other day).  Unless the partitioning process changes much, I don't think 
there is much more we can learn.  The developers just have to implement it 
all.

I would suggest picking one of the Desktop Experiences the development team is 
focused on: https://wiki.ubuntu.com/DesktopTeam/Experiences.  Doing this 
would also gain attention and support from Ubuntu.

Once we figure out what we are going to test, I will write a testing script 
and whatever questionnaires we will use to gather data, in addition to the 
other documentation such as participant consent form.

Who to test:

Friends and family are probably the easiest audience to get a hold of, and the 
most diverse.  We all have geek friends who might not yet use Linux or Moms 
who just need an email client and web browser.

Moderating:

Before hand, I would suggest whoever is interested in moderating get together 
to run through the test and practice on each other.  In theory, moderating is 
easy, but it can be intimidating the first time you do it.  Also, I want to 
give you some pointers on how to interact with participants so you don't lead 
them to the answer, feel comfortable with them struggling with a task, and 
know how to help them go in the right direction.

I would also suggest we do this in teams of two.  One person leads the 
participant through the script and the second person takes notes.  Since the 
second person will most likely be in the same room, it will be very important 
that they sit off to the side or behind the participant and moderator to take 
notes.  It is also very important they ONLY take notes and not talk with the 
participant, otherwise you get a double-teaming effect on the participants 
behavior.

Equipment:

I would suggest a laptop with an external Monitor and keyboard.  This will 
give us ultimate mobility but provide a level of comfort to the participant.  
If possible, also a KVM so an observer can sit and take notes without being 
interruptive.

Where:

1) Since it will probably be a friends+family recruit, we can just go to 
people's houses.  The problem is that the LoCo member associated with that 
participant would have to be the moderator.

2) The library is another possibility.  We could schedule a room and set up a 
computer.  The benefit is that other people's friends and family can show up.  
The only problem is our participants will have to come.

Stipends:

Even though this will probably be a friends+family recruit it is always nice 
to provide some kind of gift for the participant.  Money is expensive for us 
and tacky for family, so I think the best bet would be to get CDs or swag as 
the gift.  If this is the way we want to go, I will contact Ubuntu and see 
what they will give us.

I hope this doesn't sound too hard, it is just a lot of careful planning.

Cheers, 

~ Celeste

-- 
Celeste Lyn Paul
KDE Usability Project & HCI Working Group
usability.kde.org