Please lift aging requirement on gce-compute-image-packages in the stable release exception

Tue Aug 7 18:41:31 UTC 2018

Hi Robie,

On Mon, Aug 06, 2018 at 03:35:17PM +0100, Robie Basak wrote:
> On Thu, Jul 26, 2018 at 11:05:11PM +0800, Balint Reczey wrote:
> > Robie Basak pointed out that in the stable release exception [1] for
> > gce-compute-image-packages the wording does not explicitly lift the
> > requirement for the the package to reach the minimal age (7 days)
> > before it can migrate from -proposed to the release.

> > In practice requiring 7 days to be spent in -proposed for this source
> > package is not particularly useful since the binary packages are
> > maintained only for Google's infrastructure and both Google and us
> > test them as part of the verification process. No one else is expected
> > to test those packages in the remaining time of default required
> > minimal age.

> Thank you for raising this here. It's certainly undesirable for there to
> be ambiguity about whether the aging period applies to this package or
> not.

> I think your argument could equally be made for any package in the
> archive with an interested upstream that performs SRU verification for
> us. I don't see any reason here to make an exception just for this
> package. Without such a reason, if it makes sense to drop the
> requirement for this package, then it follows that it should make equal
> sense to change SRU policy to drop the requirement for any package where
> an involved upstream with a good track performs SRU verification for us.

> I've always seen the aging period as an opportunity for interested
> _users_ to nack a proposed SRU that breaks them - for example if their
> use case isn't covered by the SRU verification performed by others. I
> believe that I have seen this happen in practice. Perhaps not with this
> package in particular, but I think the nature of regressions is that it
> isn't practical to gather a sufficient sample size looking at a single
> package to make this kind of policy decision. One regression is too
> many.

That is certainly the purpose of the aging period, but I think this
overstates its value in the SRU process overall.  You say you believe you
have seen this happen in practice, to block a buggy SRU before it reached
-updates.  But broadly speaking:

 - dist-upgrading (or trying to use update-manager) with -proposed is a bad
   idea for a production machine, because packages in -proposed are known
   not to be consistently installable throughout the SRU lifecycle
 - therefore, running with -proposed enabled is not something that is
   broadly socialized (nor should be) as a way for users to contribute to
   the quality of Ubuntu
 - therefore, random user testing coverage of SRUs is sparse, whether we
   wait 7 days or 30.

There is *some* value to an aging period, which is why I have never
advocated dropping the default aging period entirely.  But the value is
small, not straightforwardly quantifiable, and doesn't outweigh the cost in
human developer time of multiple round-trips with the SRU team; so whenever
anyone asks to waive the waiting period, and I don't have /specific/
concerns about the SRU, my answer will be yes.

I note that there's no language at all in the wiki page at
https://wiki.ubuntu.com/StableReleaseUpdates which talks about waiving the
waiting period.  I think that is an oversight; I recall seeing written
language at one point about this, but I can't find it now (including in the
history of this wiki page).  Other long-time members of the SRU team can
check me on this, but my understanding of the actual policy is that any
member of the SRU team has the authority to waive the 7-day period any time
they think it is appropriate to do so.  I think we should fix the wiki
documentation to match.

> In this case, I think user-based regression alert opportunity still
> applies. Presumably the reason for this package to exist in the Ubuntu
> archive is that there are users. Removing the aging requirement removes
> the opportunity for those users to tell us about problems.

The package in question, gce-compute-image-packages, is only installed in
cloud instances.  I believe this means that the organic regression test
coverage by users is LESS than it would be for another package, because the
test coverage we do get tends to be from individually-deployed machines (a
personal desktop/laptop or a one-off server), which is less likely to happen
in a cloud where deployments are intended to be done at scale.[1]

And in cases where we already have an SRU exception, which dictates that
there must be strong upstream QA for the package before it is pushed to the
SRU queue, I think it's all the more reasonable to omit the default aging
period as part of that SRU exception.

> My conclusions:

> 1. In my opinion, there isn't a sufficient justification here (or at
> least presented here) to drop the aging requirement for this package.

> 2. If we do decide to make an exception for this package, we should
> consider whether the underlying reason should apply to all packages with
> similar workflows by policy, and therefore whether it should be a change
> to general policy instead.

> 3. Removing ambiguity in policy is useful. We should update the SRU
> exception documention for this package to make it clear that the aging
> requirement still stands.

> I welcome opposing opinions from the SRU team.

Thanks,
-- 
Steve Langasek                   Give me a lever long enough and a Free OS
Debian Developer                   to set it on, and I can move the world.
Ubuntu Developer                                   https://www.debian.org/
slangasek at ubuntu.com                                     vorlon at debian.org

[1] The counterpoint to this is that large cloud customers could run their
own canary instances with -proposed enabled, for purposes of doing CI of
SRUs for their own workloads.  However, if there are customers doing this
today, I haven't seen any evidence of this on SRU bugs.  If such canarying
is happening, I would welcome those cloud customers to reach out to us, and
it would carry weight with me for retaining an aging period for such
packages.  OTOH, I also expect such canarying to be extensively automated,
meaning a 7-day waiting period is still overkill for such scenarios.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: not available
URL: <https://lists.ubuntu.com/archives/ubuntu-release/attachments/20180807/1cec3cb8/attachment.sig>