Raring Cadence Week 1

[Nicholas Skaggs]

Just an update to talk about the unity autopilot tests I included as 
part of our testing this week based on feedback from the initial folks 
who have attempted to run and report results.

First, please don't file bugs against unity for any test failures you 
may encounter. Because the testsuite is still new, there are some tests 
that may fail -- don't be alarmed if you have many failures. In order to 
generate useful data, I will be manually collating the testing data from 
this week and passing the information along to the unity team. I repeat, 
please don't file bugs for now -- this won't be helpful to the unity team.

Because the testsuite is so long to run in one sitting, the testcase has 
been updated to explain how to run sections of the suite, as well as how 
to log the results from each testcase by attaching a link to your log 
via paste.ubuntu.com (or your favorite paste site :-) ). For example, 
I've posted a couple results from me personally running the tests here:

http://packages.qa.ubuntu.com/qatracker/milestones/246/builds/28425/testcases/1466/results

Switcher, PASSED:

http://paste.ubuntu.com/1390078/

Quicklists, FAILED:
http://paste.ubuntu.com/1390112/

Since your not reporting any bugs, you'll have to mark everything as 
'passed' (sadface :-( ), so be sure to post your failures in front of 
the log when you file your results. Longer term, better reporting for 
these autopilot testcases will be explored, but for now I'm enjoying 
generating some logs and seeing what autopilot is capable of. Just 
remember the more people who generate test results, the more work I get 
to do on collating them.. haha.. Make me sweat!

Nicholas

P.S. Keep the feedback coming. I want to get people familiar with the 
idea of running these autopilot testcases -- as after all, this cycle 
we're going to make more :-)

On 11/25/2012 11:56 PM, Nicholas Skaggs wrote:
> Ok, so we're a bit delayed in starting this because, well, there was a 
> Holiday in North America called Thanksgiving. Forgive me. :-p
>
> So, this week (through Saturday) is our first cadence week. What that 
> means is we will be focusing testing specific packages (or isos) 
> during the week. What you can do to help is to run through a testcase 
> (or a few :-) ) at some point during the next week and report your 
> results. The goal here is NOT to make all the tests green everyday. 
> Instead, we want to test on a deeper level and really look for bugs in 
> the software (or even the testcases :-) ) so we can be confident in 
> the health of the package(s) we are testing. To that end, there is no 
> daily target -- rather we want to test through Saturday (whenever that 
> is in your localtime ;-) ). Think quality, not quantity (I know, it's 
> a bad pun.. quality quality...)
>
> What this means practically is during the next week, try to execute 
> some testcases for our week 1 targets (which are libreoffice, unity 
> autopilot tests and the daily raring iso). You can find the wiki page 
> detailing this here:
>
> https://wiki.ubuntu.com/QATeam/Cadence/Raring/Week1
>
> The master schedule can be found here:
>
> https://wiki.ubuntu.com/QATeam/Cadence/Raring
>
> Again, I would suggest finding a day and time that works for you and 
> adding your results. Feel free to use IRC (#ubuntu-quality on 
> freenode) or this mailing list to discuss any bugs you may find. At 
> the end of the week, I'll recap what we did (bugs, tests ran, etc), 
> and we can evaluate the results to see if further followup and focus 
> is needed or not. Now, our testing focus's each week will change in 
> order to target packages that need more QA work, or new packages that 
> recently landed in QA.
>
> Now, this week I will also be posting the next bits in our autopilot 
> series -- intending to teach you about running and writing autopilot 
> testcases. Our cadence testing this cycle will include running both 
> automated and manual tests -- we need both ;-) You'll note the 
> autopilot unity testsuite is included this week. I hope our subsequent 
> testing weeks can feature some new automated autopilot tests written 
> by some of you :-) If writing automated tests aren't up your alley, 
> don't worry we need manual testcases too!
>
> Happy Testing everyone,
>
> Nicholas
>
>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.ubuntu.com/archives/ubuntu-quality/attachments/20121126/4e452735/attachment.html>