Running autopkgtests from PPA on real infrastructure

[Dan Streetman]

On Wed, Sep 16, 2020 at 12:07 PM Olivier Tilloy
<olivier.tilloy at canonical.com> wrote:
>
> On Wed, Sep 16, 2020 at 5:13 PM Iain Lane <laney at ubuntu.com> wrote:
> >
> > On Tue, Sep 15, 2020 at 03:53:01PM +0200, Lukas Märdian wrote:
> > > * Upload your package (incl. debian/tests/*) to your PPA
> > > * Get a core-dev/MOTU to trigger the test for you, via this URL scheme:
> > > https://autopkgtest.ubuntu.com/request.cgi?release=RELEASE&arch=ARCH&package=SRCPKG
> > > *&ppa=LPUSER/PPA*&trigger=SRCPKG/VERSION
> > > * Check the results via this URL scheme:
> > > https://objectstorage.prodstack4-5.canonical.com/v1/AUTH_77e2ada1e7a84929a74ba3b87153c0ac/autopkgtest-RELEASE-LPUSER-PPA/
> >
> > I would dearly love to improve this situation. I could imagine (easier)
> > a CLI tool to help you
> >
> >   (1) submit requests,
> >
> >   (2) view them [the swift software we use to store/retrieve results has
> >   an API],
>
> I wrote a couple of scripts that cater well for my particular use
> cases: I build multiple versions of firefox, thunderbird and
> chromium-browser in various PPAs, and I wanted to be able to request
> tests for a particular version prefix (potentially spanning several
> releases) in a given PPA, and to quickly view a summary of the test
> results.
> The code is at https://code.launchpad.net/~osomon/+git/ubuntu-scripts,
> and the interesting scripts are request-autopkgtests.py and
> display-filtered-autopkgtests-results-summary.py.
>
> Example use case:
>
>   ./request-autopkgtests.py ppa:ubuntu-mozilla-security/ppa firefox 81
>   # after all the tests have completed (check periodically at
> http://autopkgtest.ubuntu.com/running)
>   ./display-filtered-autopkgtests-results-summary.py
> ppa:ubuntu-mozilla-security/ppa firefox bionic 20200916
>
> I use those scripts daily and they're a big time saver.
> Suggestions and improvements welcome, and I'm happy to work towards
> having them owned and maintained by ~ubuntu-dev if they can be of use.

We (support) have a similar script; since nobody on my team has bileto
access, autopkgtests from ppas would be essentially unusable to us (or
at least, me) without this scripting.
https://git.launchpad.net/ubuntu-support-tools/tree/autopkgtest-manager/autopkgtest-manager

However, I think the real problem is autopkgtest-cloud, which is in
desperate need of updates to provide not only access to ppa-run tests,
but better ways to view test results (probably in addition to
improvements to autopkgtest itself). I did set up my own
autopkgtest-cloud instance a while ago - which was a lot harder than i
expected - but haven't had time to work on improving it since. I
wonder if it's worth improving autopkgtest-cloud, or if we should
think about moving to something else like Jenkins?

>
> > or (harder, better) an extension to the web frontend so that PPA results
> > are displayed like distro ones.
> >
> > Just mentioning this in case anyone gets excited about helping with the
> > tooling. I'd love to put this on my list, but someone else picking it up
> > would make it happen sooner. :)
>
> Cheers,
>
>  Olivier
>
> --
> ubuntu-devel mailing list
> ubuntu-devel at lists.ubuntu.com
> Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel