Please lift aging requirement on gce-compute-image-packages in the stable release exception

[Robie Basak]

Hi Steve,

Summary: I still think we should have an actual justification for
waiving the aging requirement. I still don't think that we've actually
been given a justification in this case. Give me one and I'll
reconsider, but IMHO without any justification, we should either change
policy to drop it wholesale, or maintain our current policy of having
it.

The rest of my response is mostly about why we have the general policy
in the first place, rather than this specific case.

On Tue, Aug 07, 2018 at 11:41:31AM -0700, Steve Langasek wrote:
> That is certainly the purpose of the aging period, but I think this
> overstates its value in the SRU process overall.  You say you believe you
> have seen this happen in practice, to block a buggy SRU before it reached
> -updates.  But broadly speaking:
> 
>  - dist-upgrading (or trying to use update-manager) with -proposed is a bad
>    idea for a production machine, because packages in -proposed are known
>    not to be consistently installable throughout the SRU lifecycle
>  - therefore, running with -proposed enabled is not something that is
>    broadly socialized (nor should be) as a way for users to contribute to
>    the quality of Ubuntu
>  - therefore, random user testing coverage of SRUs is sparse, whether we
>    wait 7 days or 30.

Nevertheless, I think that wider random user testing coverage of
packages in proposed should be something we aim to achieve. This is
something I've mentioned before[1]. I'm confident that we have enough of
a community willing to live on the edge that they would be willing to do
this: if we make it easy for them and publish this as "a way to help
out". Regardless of whether or not we can do this today, removing the
aging period moves us in the opposite direction.

> There is *some* value to an aging period, which is why I have never
> advocated dropping the default aging period entirely.  But the value is
> small, not straightforwardly quantifiable, and doesn't outweigh the cost in
> human developer time of multiple round-trips with the SRU team; so whenever
> anyone asks to waive the waiting period, and I don't have /specific/
> concerns about the SRU, my answer will be yes.

I disagree. Since aging is something we consider worthy enough to have
in our policy by default, I think that waiving the aging period in a
specific case should require at least _some_ justification. "I want to
do it" or "It isn't needed" is, IMHO, not in itself a valid reason.
Accepting those statements would in practice mean that aging doesn't
exist in the SRU process for all those who disagree with it; in that
case, why have an aging period in our policy at all? If we're going to
accept that, we might as well remove the requirement entirely.

On the other hand, give me an actual justification and I'd be happy to
consider it. I just don't feel like we've been given a justification at
all in this case, apart from "I don't like it".

As for requiring an extra round-trip with the SRU team, I don't think
this applies to SRUs in -proposed for which all the bugs are green and
the SRU process properly followed in the SRU report by the time the
aging period expires. In practice, I see these all being released to
-updates promptly and without further interaction.

Most delays are caused by Ubuntu developers not following documented SRU
procedure, and it is that which causes extra round trips. That can be
fixed by uploading Ubuntu developers easily enough; I don't think it's
necessary to compromise quality (since we both agree that there is
*some* value to an aging period) to do it.

> I note that there's no language at all in the wiki page at
> https://wiki.ubuntu.com/StableReleaseUpdates which talks about waiving the
> waiting period.  I think that is an oversight; I recall seeing written
> language at one point about this, but I can't find it now (including in the
> history of this wiki page).  Other long-time members of the SRU team can
> check me on this, but my understanding of the actual policy is that any
> member of the SRU team has the authority to waive the 7-day period any time
> they think it is appropriate to do so.  I think we should fix the wiki
> documentation to match.

My understanding matches yours. I have always understood any single SRU
team member to have the delegated authority to waive the aging
requirement on a case-by-case basis. What I think we're discussing here
is when the SRU team believes they should.

> > In this case, I think user-based regression alert opportunity still
> > applies. Presumably the reason for this package to exist in the Ubuntu
> > archive is that there are users. Removing the aging requirement removes
> > the opportunity for those users to tell us about problems.
> 
> The package in question, gce-compute-image-packages, is only installed in
> cloud instances.  I believe this means that the organic regression test
> coverage by users is LESS than it would be for another package, because the
> test coverage we do get tends to be from individually-deployed machines (a
> personal desktop/laptop or a one-off server), which is less likely to happen
> in a cloud where deployments are intended to be done at scale.[1]

Users managing cloud deployments done at scale following best practice
already have the ability to test such deployments automatically,
including packages in proposed, at virtually no extra maintenance cost.

That we currently don't recommend that users enable -proposed wholesale
is a use case that, in my view, we should fix (whether by "fixing"
-proposed or by providing some other mechanism). I see the current
inability as a "bug", and I don't think the existence of this bug should
be used as a justification for removing the aging period; that would
make it impossible to solve the problem.

> And in cases where we already have an SRU exception, which dictates that
> there must be strong upstream QA for the package before it is pushed to the
> SRU queue, I think it's all the more reasonable to omit the default aging
> period as part of that SRU exception.

Strong upstream QA certainly does shift the balance of regression risk
in the "safer" direction. However, packages with SRU exceptions
generally end up with riskier stable uploads, such as feature additions.
So that's the counterpoint that could shift the balance back and cancel
this out. Going in the other direction, it might well be a reasonable
mitigation for the extra regression risk in permitting wholesale
upstream feature releases into Ubuntu stable releases to require a
_longer_ aging period. I'm not suggesting we do this, but I hope this
example helps illustrate my point.

Robie

[1] https://lists.ubuntu.com/archives/ubuntu-release/2016-October/003948.html
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: not available
URL: <https://lists.ubuntu.com/archives/ubuntu-release/attachments/20180807/2e417e19/attachment.sig>