[QA] Status Report 2012-12-07

Fri Dec 7 22:06:17 UTC 2012

Hi all,

we didn't go back into the swing of sending reports after UDS, so here
is one on behalf of QA:

=== Release Testing ===

==== Raring Testing ====
*Builds*
http://reports.qa.ubuntu.com/reports/isotesting/builds/raring.html

*Testing*
http://reports.qa.ubuntu.com/smoke/raring/flat

[Smoke testing]
#1085918 in Ubuntu CD Images: "Chinese raring desktop images have
different format of media-info to that of standard desktop images"
#1085961 in ubiquity (Ubuntu): "Unable to install two raring desktops
side-by-side"
#1086339 in ubiquity (Ubuntu): "Can not install raring desktop with
manual partitioning"
#1087630 in linux (Ubuntu): "i386 server minimal virtual installations
are bloated"

[Upstream testing]
#1084996 in apport (Ubuntu): " StacktraceAddressSignature is missing"
#1086309 in apport (Ubuntu): "expose exceptions from hooks"

[Kernel SRU testing]
#1087009 in QA Regression Testing: "python backtrace on
test-kernel-security on Raring"
#1087354 in ubuntu-kernel-tests: "kernel-testing: apparmor failures due
to missing pre-req"

==== Precise Testing ====
*Builds*
http://reports.qa.ubuntu.com/reports/isotesting/builds/precise.html

*Testing*
http://reports.qa.ubuntu.com/smoke/precise/flat

[Smoke testing]
#1087241 in ubuntu-server-iso-testing: "The test code needs to be
changed to accommodate the renaming of vmlinuz to vmlinuz.efi for amd64
desktop kernels"

==== UTAH Testing ====
Since we are running Raring smoke testing with UTAH, it becomes critical
to have the tool in top condition every day, and fix issues that may
affect smoke testing really quickly. The Plarform QA Team is dealing
with these bugs. Here is the list for this week:

[UTAH]
#1085835 in UTAH: "ceph smoketest jobs do not honore timeout"
#1086772 in UTAH: "iso_static_validation tests needs to change due to
the renaming of vmlinuz to vmlinuz.efi"
#1087233 in UTAH: "UTAH server is not able to connect to the VM client
for raring desktop images"
#1087288 in UTAH: "'UTAHProvisioningException: Failed to install client'
on raring server provisioning after update to
utah-client_0.5ubuntu15-r769~precise1_all.deb"
#1087620 in UTAH: "vmlinuz presence should not be tested for server
images in static validation"
#1087679 in UTAH: "UTAH fails to install oem with raring desktop"
#1087312 in UTAH: "Please save out the preseed by default"
#1087237 in UTAH: "Kernel path detection in image files"
#1084655 in UTAH: "logs after a reboot are split into multiple yaml
documents"
#1087310 in UTAH: "Add client log schema validation"
#1086443 in UTAH: "UTAH does not gather syslogs for bare metal
provisioning after install"
#1086445 in UTAH: "UTAH does not report progress for ongoing installs"
#1086447 in UTAH: "UTAH should report information about the process that
is locking a physical machine"
#1086450 in UTAH: "run_utah_tests does not handle reboots properly"
#1087384 in UTAH: "ISO class cannnot download development version
without dl-ubuntu-test-iso"

=== What was done engineering wise? ===

Since the beginning of the cycle we have been working on moving our
smoke testing from the scripts used before to UTAH. This allows us much
more flexibility to add/remove test cases from the runs and avoids test
code duplication.

We have also put together an improved version of the QA Dashboard, that
feeds directly from jenkins[1] and gives an overview of broken
images/broken testing.

Up until now it was not just Raring images broken every now and then,
but also the testing itself, that is why there is so much red and orange
in the past weeks:

http://reports.qa.ubuntu.com

Any feedback on the reporting will be very welcome, we are happy to
answer any questions you may have or even organize a meeting to discuss
it if that is felt necessary.

=== What's about to land that might impact the other teams and release
as a whole? ===

Bootspeed testing is also being reworked and reporting is about to land.

=== Summary of bugs working on by team (reasonably reliable) ===

http://reports.qa.ubuntu.com/reports/qa/qa-open-bugs-raring.html
http://reports.qa.ubuntu.com/reports/qa/qa-open-bugs.html

=== Dependencies on other teams to make deliverables, blocking items,
release wide concerns? ===

Please, keep adding test cases so that we can increase coverage !

=== Team Status ===
http://status.ubuntu.com/ubuntu-quantal/canonical-platform-qa.html

Thanks,
Gema

[1] https://jenkins.qa.ubuntu.com/view/Raring/view/Smoke%20Testing/

-- 
Gema Gomez-Solano        <gema.gomez-solano at canonical.com>
Ubuntu QA Team           https://launchpad.net/~gema.gomez
Canonical Ltd.           http://www.canonical.com