Cadence Testing For Saucy (Craig Hrabal)

[Craig Hrabal]

The mockups are pretty excellent.  I would argue that the second choice 
is better, and combining them into one looks better visually.

I think the "packages we care about" list should refer mainly to default 
pre-installed packages within Ubuntu, obviously with a few exceptions, 
as the intent is to make sure the packages that will ship by default in 
saucy are as stable as possible.

+1 from me.

-Craig Hrabal
>
>
>
>
> Message: 5
> Date: Tue, 21 May 2013 17:48:22 -0400
> From: Nicholas Skaggs <nicholas.skaggs at canonical.com>
> To: "ubuntu-quality at lists.ubuntu.com"
> 	<ubuntu-quality at lists.ubuntu.com>
> Subject: Cadence Testing for Saucy
> Message-ID: <519BEBA6.5010108 at canonical.com>
> Content-Type: text/plain; charset="iso-8859-1"; Format="flowed"
>
> So vUDS is behind us and it's time to solidify the cadence testing
> schedule for Saucy. I've update the cadence page with actual dates now,
> starting June 15th. See the schedule here:
> https://wiki.ubuntu.com/QATeam/Cadence/Saucy
>
> Now in addition to that, as part of the
> https://blueprints.launchpad.net/ubuntu/+spec/community-s-quality-coverage
> blueprint we discussed the idea brought up by crhrabal and smartboyhw
> (thanks guys!). The outcome was an idea to change the way we do cadence
> testing. The iead was to track all the packages that we care about for
> the entire cycle -- things like our list of default applications
> firefox, thunderbird, nautilus, etc. As a new build of the package is
> published to the archive a new build is entered into the tracker and all
> subscribers to that package are notified. I promised to mock up the
> idea, and that's what I'm including below for discussion :-)
>
> Let's step back quickly for a moment though. For those not familiar with
> last cycle's cadence testing, let me describe it quickly. Every cadence
> week we created a milestone and chose packages to test. In addition we
> always tested the daily images during that week, as well as sometimes
> including a bit of hardware testing against the milestone. The cadence
> milestone was only open for the cadence week, after which the results
> would be frozen.
>
> Onto the mockups for the new idea! I've laid out two examples of how we
> could implement the new idea.
>
> The first shows the idea of lumping all packages into one milestone;
> http://packages.qa.dev.stgraber.org/qatracker/milestones/252/builds. If
> you then view the history
> http://packages.qa.dev.stgraber.org/qatracker/milestones/252/history you
> can see every package we're tracking, test results, and bugs. Clicking
> on any old build let's you see the details as well.
>
> The second shows the idea of giving each package a milestone;
> http://packages.qa.dev.stgraber.org/qatracker/milestones/253/builds. If
> you then view the history
> http://packages.qa.dev.stgraber.org/qatracker/milestones/253/history you
> can see only that package, test results, and bugs. Clicking on any old
> build let's you see the details as well.
>
> So what does this new idea do for us?
> -- Let's us follow a package for the entire cycle, and provides bugs
> linked to versions, and allows you to 'track' the status of the package
> in ubuntu
> -- Provides a summary report of bugs specific to that package that we've
> opened
> -- Allows you to subscribe to a package you like/care about and make
> sure it's tested
> -- Allows you to filter test results / versions / bugs by time
>
> What I'm looking to gather now is if we should switch how we test our
> packages as part of our cadence testing to the new system. Let me
> describe how it would work.
>
> Each cadence week we would:
> -- Test the daily images
> -- (Optionally, when requested) Perform laptop/hardware tests against
> specific image
> -- Test the packages we're tracking and ensure results are entered for
> the current builds
>
> The difference is that the milestones would be availible outside of the
> 'designated' cadence weeks and thus you are free to test the packages at
> any time, as always, but you can also now report your results! The
> cadence weeks stay a rallying cry towards us committing to test
> regularly to ensure the archive, images and packages are in good shape
> all throughout the cycle.
>
> So, in summary, let's hear your feedback on:
>
> 1) Switching to the new idea for tracking packages all cycle
> 2) Lumping the packages together or making a milestone for each one
>
> If we do decide to switch, we'll need to create a list of "packages we
> care about" :-)
>
> Nicholas
> -------------- next part --------------
> An HTML attachment was scrubbed...
> URL: <https://lists.ubuntu.com/archives/ubuntu-quality/attachments/20130521/0726ad4f/attachment.html>
>
> ------------------------------
>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.ubuntu.com/archives/ubuntu-quality/attachments/20130522/1cae2f1d/attachment-0001.html>