Juju SRU and QA plans

[Steve Langasek]

Hi Robie,

On Tue, Oct 07, 2014 at 10:21:44AM +0100, Robie Basak wrote:
> First, an explanation of why I want the process to be different.

> In Ubuntu, we want QA on binaries in trusty-proposed before accepting
> them into trusty-updates. If a bug is found, then we will remove the
> binary from trusty-proposed and not publish it in trusty-updates.

> Juju upstream want to consider any bug that prevents inclusion into
> trusty-updates as an upstream release blocking bug. That is: upstream do
> not want to release something that would subsequently fail to enter
> trusty-updates.

> So there is an ordering issue here. With the normal process, we'd
> consider entry into trusty-updates after an upstream release. But here,
> upstream don't want to release until the update has passed Ubuntu's QA.

Given that Juju is covered by a provisional micro release exception[1], I
don't understand the ordering concern.  MREs are granted on the condition
that the upstream release process is sufficient to QA the update, without a
separate verification step in Ubuntu.  If this QA is happening as expected,
what problems would you expect to turn up at the SRU verification stage that
wouldn't have already been caught by upstream's own testing?

   https://wiki.ubuntu.com/StableReleaseUpdates/MicroReleaseExceptions

> A Juju upstream release follows a step that is relevant to us here,
> which is that upstream also now have a -proposed equivalent.

> Can we tie these together in a way that would be acceptable to our
> existing processes in Ubuntu?

> For example:

> (I'm not sure my understanding is accurate about devirt PPAs, pocket
> copies, our process around them and who has permission to do them, and
> thus if this would even work technically. Corrections appreciated.)

> What if I arrange a devirt PPA and upload to it when upstream proposes a
> release. I would target both utopic and trusty-proposed, but instead of
> uploading as I might to do the archive, I upload to this PPA instead.

> Then upstream QA (and anyone else) can test the proposed binary packages
> from this PPA to help decide whether upstream should release. If QA
> fails at this stage, upstream will not release, fix the issue and skip
> the version number for the next proposed release. In this case I'll just
> delete from the PPA.

> When upstream release, I can pocket copy (including binaries) to both
> utopic-proposed and trusty-proposed respectively. After the upload to
> trusty-proposed is accepted by the SRU team, our QA result from the
> devirt PPA should still be valid, so we should be able to mark
> verification-done immediately.

If upstream hasn't yet released, then it would seem incorrect to upload
these packages to the devirt ppa using an upstream release version number.
In that case, you would still need a second package upload/build once the
upstream release has happened.  So I don't think a pocket copy here would
help any.

For the record, yes it's technically possible for you to do a pocket copy
from a devirt ppa to the trusty-proposed unapproved queue.  Last I checked,
though, Launchpad still wasn't able to give us review diffs for pocket
copies; so between this and the need to respin when the release happens, I
don't recommend it here.

Cheers,
-- 
Steve Langasek                   Give me a lever long enough and a Free OS
Debian Developer                   to set it on, and I can move the world.
Ubuntu Developer                                    http://www.debian.org/
slangasek at ubuntu.com                                     vorlon at debian.org
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: Digital signature
URL: <https://lists.ubuntu.com/archives/ubuntu-release/attachments/20141007/ec3f7e47/attachment.pgp>