Better targeted ISO testing

[Martin Pitt]

Hi Henrik,

Henrik Nilsen Omma [2007-10-10 21:12 +0200]:
> In the first stage we should aim for the widest possible code coverage 
> with fewer tests and we should do this right after the milestone freeze. 
> The purpose of this is to uncover release critical bugs, not to validate 
> the CDs for release. This requires a sensible mix of archs, CD types and 
> install methods, but not full coverage of everything. Once those bugs 
> are discovered we can get to work on fixing them without being too tight 
> up against the milestone release.
> 
> By the time we start the second stage we should be more confident that 
> all the show-stoppers have been found and we can focus on real 
> validation testing on all CDs, as well as noting non-release critical 
> bugs for the future. This will involve a larger group of people and 
> would run 1-2 days.

I like this approach. Would be great to make it work like that.

The first dailies after the RC images should already have some
high-impact bug fixes, so we should base the testing on them to avoid
some duplicated reports about known issues.

For this kind of testing it is important to file detailled bug reports
about everything you consider important and subscribe (!, not assign)
"ubuntu-release", so that the release team gets aware of that bug
immediately and can give feedback, milestone it, etc. This does not
really need the iso tracker even, since we are not doing release
validation.

Martin

-- 
Martin Pitt        http://www.piware.de
Ubuntu Developer   http://www.ubuntu.com
Debian Developer   http://www.debian.org