[utah-devel] UTAH Weekly Meeting

[Gema Gomez]

Hi,

summary of our conversation about UTAH today:

*Attendees*
jcollado
nuclearbob
josepht
jamespage
gema

*Discussion*
- Bootspeed testing is running and producing good results with UTAH,
next step is to have nicer reports (outside of this forum but we will
keep you posted if you want).
- There are some failures in the UTAH testing this week (dev branch)
that nuclearbob is working on fixing.
- We are going to be publishing test results for UTAH itself to the
public instance of jenkins, so that whenever you start using the tool,
you can check if anything is broken.
- We will be adding variant (alternate/desktop/server) as a flag to aid
reporting, given that we haven't found a way to determine which kind of
install we were running post installation, suggestions welcome.
- The current JSON Schema is messy and has two different definitions, we
are going to be cleaning that up. This may break some runlists, we will
post to the list whenever it changes with the required changes to
existing runlists.
- Moving slowly away from the wiki and more into insource documentation,
utah.readthedocs.org has the lastest and the greatest doc.
- We've been discussing about testing graphical applications, and how to
go from ssh-ing into a machine to having access to a full X session
where we can execute things. We will be looking into how other people
are doing this, jamespage pointed us to OpenQA[1], we will be having a
look and also talk to other teams that are already doing this (PS with
autopilot) so see how to move forward with UI testing.

- We are working on the test cases migration, and found that template
preseeding is a very good to have tool when trying to manage many
different installs, jamespage will be extracting that code from the
current server iso testing and we will integrate that script into utah,
as a standalone script (for debugging purposes) and also as part of the
normal execution for full automation.

- jamespage has started looking into utah for the server team and he
gave us valuable user feedback that we will be acting upon this coming
weeks.
 - When something fails, right now there is a problem, we don't get any
logs (this is a bug and will be fixed as such).
 - It'd be useful if utah returned xunit output for jenkins' benefit, we
are adding a work item for this.
 - nuclearbob will be updating the utah help and documentation with the
-g option, which was missing.


This is very useful feedback for us, so if you are also trying to use
utah and have problems, please either come to the meeting to talk to us
or ping on IRC so that we can fix them.

Tomorrow, I will be talking about UTAH in the UDW[2]. There will be a
Questions and Answers half an hour for generic automation
questions/discussions. So if you have something you'd like to
discuss/ask, just come and ask.

*Summary of bugs we have been working on*
- Bug #1039540: Default runlist fails
https://bugs.launchpad.net/bugs/1039540

- Bug #1040015: Validation errors should specify which file failed
https://bugs.launchpad.net/bugs/1040015

- Bug #1040753: Extra newline characters in test case results
https://bugs.launchpad.net/bugs/1040753

- Bug #1042510: client output not available in server log
https://bugs.launchpad.net/bugs/1042510

- Bug #1043174: Desktop installation type not correctly detected in
daily image
https://bugs.launchpad.net/bugs/1043174


As usual, happy to hear any updates/corrections/contributions/questions!

Thanks,
Gema

[1] http://openqa.opensuse.org/
[2] https://wiki.ubuntu.com/UbuntuDeveloperWeek
-- 
Gema Gomez-Solano        <gema.gomez-solano at canonical.com>
Ubuntu QA Team           https://launchpad.net/~gema.gomez
Canonical Ltd.           http://www.canonical.com