TEAM: Wily Werewolf Package Testing

[Pasi Lallinaho]

>> Also, the current testcases we have for packages and iso are the
>> same. They need to change or be upgrade with new procedures. We can
>> test the packages over and over only to confirm what worked during
>> the last development cycle.
>>

How do they need to change? What are the new procedures you are
referring to?

>
>> Every cycle should have new upgraded testcases. Rather testing basic
>> functionality, the testcases should really push the system. The
>> objective should be to try and break the system then fix it.
>>

What's wrong with making sure that some basic features work with our
applications? Can we honestly make the assumption that the features that
have worked before will work in the future?

These have been broken before, and it's more likely than not that
something well break in the future. If the basic functionality breaks
for a user, what use is the application even if it can allegedly perform
advanced operations?

Sure, it is great if we can get testers do testing that actually "pushes
the system", but that can never be achieved with prewritten testcases.
Running prewritten testcases never put the system in the real test, they
will always just test things that people in the team have though that
are "worth testing" and/or basic enough to have a clearly specified
testcase.

That's what we have exploratory testing for. Due to its nature, you
can't really tell people do it though, unless they have the motivation
to run the development version. If they do, they will most likely file
the bugs that they found while doing their usual activities.

Instead of laying out a list of things we need to do, do you have any
ideas HOW we can do/achieve those things - or get started doing them?
Above, I've asked some questions that I think would help us figure these
things out.

Cheers,
Pasi

-- 
Pasi Lallinaho (knome)                » http://open.knome.fi/
Leader of the Shimmer Project         » http://shimmerproject.org/
Ubuntu member, Xubuntu Website lead   » http://xubuntu.org/