Usability Testing

[Celeste Lyn Paul]

On Monday 24 March 2008 13:07:42 Chris Gregan wrote:
> Team,
> Let me know if there is any tips I can provide to the best way of
> performing the testing. I have been a professional software tester for
> 10 years and would be happy to help in any way.
>
> Also, as the new MSG (Mobile Solutions Group) QA Engineer for Canonical,
> if any of you have an Eee or the like and want to check out the latest
> mobile OS for testing, feel free to contact me directly:
> cgregan at canonical.com.

Congratulations!  (Are you going to UDS in May)

>
>
> Chris Gregan
> Open Source Consultant/Founder
> Aptenix LLC Desktop Solutions
> New Market MD 21774
> cgregan at aptenix.com
> 240-422-9224
>
> "Open source, open minds."
>
> This message is confidential, intended only for the named recipient(s) and
> may contain information that is privileged or exempt from disclosure under
> applicable law. If you are not the intended recipient(s), you are notified
> that the dissemination, distribution, or copying of this message is
> strictly prohibited. If you receive this message in error or are not the
> named recipient(s), please notify the sender at either the fax address or
> telephone number above and delete this message. Thank you.
>
> Celeste Lyn Paul wrote:
> > Chuck forwarded me the notes from last night and we talked a little about
> > your idea for usability testing in the LoCo.  I think this is a great
> > idea if we plan it right.  I added it as an agenda item for our next IRC
> > meeting, but I also wanted to open the conversation up on the mailing
> > list.
> >
> > What to test:
> >
> > Your first reaction is probably the installation process, but to tell you
> > the truth Ubiquity has been tested to death.  It has had two usability
> > tests conducted on it and multiple UI reviews (I just submitted some UI
> > bugs the other day).  Unless the partitioning process changes much, I
> > don't think there is much more we can learn.  The developers just have to
> > implement it all.
> >
> > I would suggest picking one of the Desktop Experiences the development
> > team is focused on: https://wiki.ubuntu.com/DesktopTeam/Experiences. 
> > Doing this would also gain attention and support from Ubuntu.
> >
> > Once we figure out what we are going to test, I will write a testing
> > script and whatever questionnaires we will use to gather data, in
> > addition to the other documentation such as participant consent form.
> >
> > Who to test:
> >
> > Friends and family are probably the easiest audience to get a hold of,
> > and the most diverse.  We all have geek friends who might not yet use
> > Linux or Moms who just need an email client and web browser.
> >
> > Moderating:
> >
> > Before hand, I would suggest whoever is interested in moderating get
> > together to run through the test and practice on each other.  In theory,
> > moderating is easy, but it can be intimidating the first time you do it. 
> > Also, I want to give you some pointers on how to interact with
> > participants so you don't lead them to the answer, feel comfortable with
> > them struggling with a task, and know how to help them go in the right
> > direction.
> >
> > I would also suggest we do this in teams of two.  One person leads the
> > participant through the script and the second person takes notes.  Since
> > the second person will most likely be in the same room, it will be very
> > important that they sit off to the side or behind the participant and
> > moderator to take notes.  It is also very important they ONLY take notes
> > and not talk with the participant, otherwise you get a double-teaming
> > effect on the participants behavior.
> >
> > Equipment:
> >
> > I would suggest a laptop with an external Monitor and keyboard.  This
> > will give us ultimate mobility but provide a level of comfort to the
> > participant. If possible, also a KVM so an observer can sit and take
> > notes without being interruptive.
> >
> > Where:
> >
> > 1) Since it will probably be a friends+family recruit, we can just go to
> > people's houses.  The problem is that the LoCo member associated with
> > that participant would have to be the moderator.
> >
> > 2) The library is another possibility.  We could schedule a room and set
> > up a computer.  The benefit is that other people's friends and family can
> > show up. The only problem is our participants will have to come.
> >
> > Stipends:
> >
> > Even though this will probably be a friends+family recruit it is always
> > nice to provide some kind of gift for the participant.  Money is
> > expensive for us and tacky for family, so I think the best bet would be
> > to get CDs or swag as the gift.  If this is the way we want to go, I will
> > contact Ubuntu and see what they will give us.
> >
> > I hope this doesn't sound too hard, it is just a lot of careful planning.
> >
> > Cheers,
> >
> > ~ Celeste



-- 
Celeste Lyn Paul
KDE Usability Project & HCI Working Group
usability.kde.org