[Merge] lp:~canonical-platform-qa/webbrowser-app/autopkgtest into lp:webbrowser-app

Tue May 5 11:43:42 UTC 2015

Thanks for the review !

All fixed as mentioned in the inline comments.

Diff comments:

> === modified file 'README'
> --- README	2015-04-01 05:57:42 +0000
> +++ README	2015-04-24 12:37:09 +0000
> @@ -40,11 +40,11 @@
>  
>      $ ctest
>  
> -
>  = Automated UI tests =
>  
>  webbrowser-app uses autopilot (https://launchpad.net/autopilot) to test its UI.
> -To run the tests, you will need to install python3-autopilot and autopilot-qt5.
> +To run the tests locally, you will need to install python3-autopilot and
> +autopilot-qt5.
>  Then do the following:
>  
>      $ cd tests/autopilot/
> @@ -54,6 +54,8 @@
>  
>      $ autopilot3 list webbrowser_app
>  
> +In order to run the tests in a virtual machine with an environment closer to
> +what a user will get in Ubuntu Touch, see the Dep8 tests section.
>  
>  = Code coverage =
>  
> @@ -68,6 +70,63 @@
>  This will generate a coverage report in XML format (coverage.xml) and an
>  interactive human-readable report in HTML format (coveragereport/index.html).
>  
> += Dep8 tests =
> +
> +Dep8 tests excercise the package "as-installed".
> +
> +Currently, the webbrowser-app has one suite of dep8 tests that uses autopilot
> +(https://launchpad.net/autopilot) to test from the point of view of the user.
> +
> +To run the tests you will need autopkgtest:
> +
> +    $ sudo apt-get install autopkgtest
> +
> +You can use multiple test beds to execute the tests. Below you will find
> +instructions to run them in a virtual machine
> +You can find more information with:
> +
> +    $ man adt-run
> +
> +== Run dep8 tests ==
> +
> +To run the tests in a qemu virtual machine, first you will have to create
> +it. We output the image to ~/ rather than the current directory, so it will
> +be in a safer place to avoid rebuilding images every time. You can store it
> +in any directory you wish. This image is better consumed "fresh", building
> +it daily will avoid long updates/upgrades when running the tests.
> +
> +    $ adt-buildvm-ubuntu-cloud -r vivid -a amd64 -o ~/
> +
> +Then run the tests using adt-run with the qemu virtualization host against
> +the current archive.
> +
> +    $ adt-run -B -U --unbuilt-tree . \
> +      -o /tmp/adt-browser-test \
> +      --- qemu ~/adt-vivid-amd64-cloud.img
> +
> +To run the tests as in proposed-migration.
> +
> +   $ adt-run -B -U --unbuilt-tree . --apt-pocket proposed\
> +      -o /tmp/adt-browser-test \
> +      --- qemu ~/adt-vivid-amd64-cloud.img
> +
> +The tests can also be run on a local phone.
> +
> +    $ adt-run -B -U --unbuilt-tree . \
> +      -o /tmp/adt-browser-test \
> +      --- ssh -s adb -- -p <password> --serial <ADB_SERIAL> 
> +
> +== Examine the dep8 autopilot results ==
> +
> +To examine the test results, which are in subunit format, additional tools are
> +required.
> +
> +    $ sudo add-apt-repository ppa:thomir/trv
> +    $ sudo apt-get update
> +    $ sudo apt-get install trv
> +    $ trv
> +
> +You can find the results file in the directory /tmp/adt-browser-test/artifacts.

I added a FIXME in revno 981 to keep track of this effort.

>  
>  = Settings =
>  
> @@ -93,4 +152,3 @@
>  
>   - restoreSession: whether to restore the previous browsing session at startup
>     (defaults to true)
> -
> 
> === modified file 'debian/control'
> --- debian/control	2015-04-10 13:33:10 +0000
> +++ debian/control	2015-04-24 12:37:09 +0000
> @@ -26,6 +26,7 @@
>  # If you aren't a member of ~phablet-team but need to upload packaging changes,
>  # just go ahead.  ~phablet-team will notice and sync up the code again.
>  Vcs-Bzr: https://code.launchpad.net/~phablet-team/webbrowser-app/trunk
> +XS-Testsuite: autopkgtest

Dropped as revno 982.

>  
>  Package: webbrowser-app
>  Architecture: any
> 
> === added directory 'debian/tests'
> === added file 'debian/tests/control'
> --- debian/tests/control	1970-01-01 00:00:00 +0000
> +++ debian/tests/control	2015-04-24 12:37:09 +0000
> @@ -0,0 +1,28 @@
> +# Copyright 2015 Canonical
> +#
> +# This program is free software: you can redistribute it and/or modify it
> +# under the terms of the GNU General Public License version 3, as published
> +# by the Free Software Foundation.
> +#
> +# This program is distributed in the hope that it will be useful,
> +# but WITHOUT ANY WARRANTY; without even the implied warranty of
> +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
> +# GNU General Public License for more details.
> +#
> +# You should have received a copy of the GNU General Public License
> +# along with this program. If not, see <http://www.gnu.org/licenses/>.
> +
> +# FIXME: Specifying python3-evdev as a dep shouldn't be needed. It's a
> +# Recommends in both autopilot-touch and autopilot-desktop. No idea why it's
> +# not installed on the phone -- vila 2015-04-23

Dropped as part of revno 983 'Classes: ubuntu-touch' also added.

> +
> +# autopkgtest is a test dependency so we don't need to copy
> +# ubuntu-touch-session to the testbed but can use it from autopkgtest instead
> +
> +Test-Command: debian/tests/touch-session-autopilot webbrowser_app
> +Restrictions: allow-stderr
> +Depends: webbrowser-app-autopilot, autopkgtest, python3-evdev
> +
> +Test-Command: debian/tests/touch-session-autopilot webapp_container
> +Restrictions: allow-stderr
> +Depends: webapp-container-autopilot, autopkgtest, python3-evdev
> 
> === added file 'debian/tests/touch-session-autopilot'
> --- debian/tests/touch-session-autopilot	1970-01-01 00:00:00 +0000
> +++ debian/tests/touch-session-autopilot	2015-04-24 12:37:09 +0000
> @@ -0,0 +1,35 @@
> +#!/bin/sh
> +
> +# Copyright 2015 Canonical
> +#
> +# This program is free software: you can redistribute it and/or modify it
> +# under the terms of the GNU General Public License version 3, as published
> +# by the Free Software Foundation.
> +#
> +# This program is distributed in the hope that it will be useful,
> +# but WITHOUT ANY WARRANTY; without even the implied warranty of
> +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
> +# GNU General Public License for more details.
> +#
> +# You should have received a copy of the GNU General Public License
> +# along with this program. If not, see <http://www.gnu.org/licenses/>.
> +
> +# This runs the $1 autopilot suite ensuring that the testbed is configured
> +# properly
> +
> +SUITE=$1
> +
> +set -e
> +
> +if ! pgrep -f unity-system-compositor ; then
> +   # We're not using Mir, setup enough of an X stack
> +
> +   # FIXME: We won't need an X stack once Mir provides mirvfb or any other
> +   # mean to test consistently on all platforms -- vila 2015-04-22
> +   sudo ADT_NORMAL_USER=${USER} sh -e /usr/share/autopkgtest/setup-commands/ubuntu-touch-session

Fixed as revno 984.

> +   # Import the environment produced above in the current shell
> +   set -a
> +   . /etc/environment
> +fi
> +
> +autopilot3 run -v -f subunit -o ${ADT_ARTIFACTS} ${SUITE}
> 

-- 
https://code.launchpad.net/~canonical-platform-qa/webbrowser-app/autopkgtest/+merge/256858
Your team Ubuntu Phablet Team is subscribed to branch lp:webbrowser-app.