Cadence Testing For Saucy (Craig Hrabal)

[Elfy]

On 22/05/13 19:34, Craig Hrabal wrote:
> The mockups are pretty excellent.  I would argue that the second 
> choice is better, and combining them into one looks better visually.
>
> I think the "packages we care about" list should refer mainly to 
> default pre-installed packages within Ubuntu, obviously with a few 
> exceptions, as the intent is to make sure the packages that will ship 
> by default in saucy are as stable as possible.
>
> +1 from me.
>
> -Craig Hrabal

I'd suspect that not everyone will feel quite the same - other than us 
/all/  releasing good systems to the world at large, I'd much prefer 
that the packages we care most about related to Xubuntu ;)

>>
>>
>> Message: all
>> Date: Tue, 21 May 2013 17:48:22 -0400
>> From: Nicholas Skaggs<nicholas.skaggs at canonical.com>
>> To:"ubuntu-quality at lists.ubuntu.com"
>> 	<ubuntu-quality at lists.ubuntu.com>
>> Subject: Cadence Testing for Saucy
>> Message-ID:<519BEBA6.5010108 at canonical.com>
>> Content-Type: text/plain; charset="iso-8859-1"; Format="flowed"
>>
>> So vUDS is behind us and it's time to solidify the cadence testing
>> schedule for Saucy. I've update the cadence page with actual dates now,
>> starting June 15th. See the schedule here:
>> https://wiki.ubuntu.com/QATeam/Cadence/Saucy
>>
>> Now in addition to that, as part of the
>> https://blueprints.launchpad.net/ubuntu/+spec/community-s-quality-coverage  
>> blueprint we discussed the idea brought up by crhrabal and smartboyhw
>> (thanks guys!). The outcome was an idea to change the way we do cadence
>> testing. The iead was to track all the packages that we care about for
>> the entire cycle -- things like our list of default applications
>> firefox, thunderbird, nautilus, etc. As a new build of the package is
>> published to the archive a new build is entered into the tracker and all
>> subscribers to that package are notified. I promised to mock up the
>> idea, and that's what I'm including below for discussion :-)
>>
>> Let's step back quickly for a moment though. For those not familiar with
>> last cycle's cadence testing, let me describe it quickly. Every cadence
>> week we created a milestone and chose packages to test. In addition we
>> always tested the daily images during that week, as well as sometimes
>> including a bit of hardware testing against the milestone. The cadence
>> milestone was only open for the cadence week, after which the results
>> would be frozen.
>>
>> Onto the mockups for the new idea! I've laid out two examples of how we
>> could implement the new idea.
>>
>> The first shows the idea of lumping all packages into one milestone;
>> http://packages.qa.dev.stgraber.org/qatracker/milestones/252/builds. If
>> you then view the history
>> http://packages.qa.dev.stgraber.org/qatracker/milestones/252/history  you
>> can see every package we're tracking, test results, and bugs. Clicking
>> on any old build let's you see the details as well.
>>
>> The second shows the idea of giving each package a milestone;
>> http://packages.qa.dev.stgraber.org/qatracker/milestones/253/builds. If
>> you then view the history
>> http://packages.qa.dev.stgraber.org/qatracker/milestones/253/history  you
>> can see only that package, test results, and bugs. Clicking on any old
>> build let's you see the details as well.
>>
>> So what does this new idea do for us?
>> -- Let's us follow a package for the entire cycle, and provides bugs
>> linked to versions, and allows you to 'track' the status of the package
>> in ubuntu
>> -- Provides a summary report of bugs specific to that package that we've
>> opened
>> -- Allows you to subscribe to a package you like/care about and make
>> sure it's tested
>> -- Allows you to filter test results / versions / bugs by time
>>
>> What I'm looking to gather now is if we should switch how we test our
>> packages as part of our cadence testing to the new system. Let me
>> describe how it would work.
>>
>> Each cadence week we would:
>> -- Test the daily images
>> -- (Optionally, when requested) Perform laptop/hardware tests against
>> specific image
>> -- Test the packages we're tracking and ensure results are entered for
>> the current builds
>>
>> The difference is that the milestones would be availible outside of the
>> 'designated' cadence weeks and thus you are free to test the packages at
>> any time, as always, but you can also now report your results! The
>> cadence weeks stay a rallying cry towards us committing to test
>> regularly to ensure the archive, images and packages are in good shape
>> all throughout the cycle.
>>
>> So, in summary, let's hear your feedback on:
>>
>> 1) Switching to the new idea for tracking packages all cycle
>> 2) Lumping the packages together or making a milestone for each one
>>
>> If we do decide to switch, we'll need to create a list of "packages we
>> care about" :-)
>>
>> Nicholas
>> -------------- next part --------------
>> An HTML attachment was scrubbed...
>> URL:<https://lists.ubuntu.com/archives/ubuntu-quality/attachments/20130521/0726ad4f/attachment.html>
>>
>> ------------------------------
>>
>
>
>
I'm in favour of the more detailed options. I assume that to be 
milestones for each package we want to follow.

I'm in favour of switching to this idea.

I assume that we'll be able to tailor the 'packages we care about' to 
our own requirements.


Elfy

-- 
Ubuntu Forum Council Member

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.ubuntu.com/archives/ubuntu-quality/attachments/20130522/4691a4bd/attachment.html>