Please lift aging requirement on gce-compute-image-packages in the stable release exception

Wed Aug 8 14:10:26 UTC 2018

Hello,

In my humble opinion, for this particular case I would be inclined
toward lifting the aging requirement. Yes, it's not a completely
straightforward thing, even though I usually approved of releasing
early of this package before after validation was performed. From my
point of view a strong enough rationale has been provided for that.
Balint's original e-mail states:
"No one else is expected to test those packages in the remaining time
of default required minimal age."
I think in case of this package this is true. As already mentioned by
Steve, the gce-compute-image-packages binaries are *only* used on
cloud instances. This basically means almost absolutely no one will be
using the versions from -proposed, meaning the 7 days are wasted on
idle waiting. My rationale below:

To me the aging period is always a safety buffer for when a package
enters -proposed, during which it can get additional testing (or
'dogfooding') by both interested and unrelated parties. In my eyes
though, usually what this means in practice is to give users that are
affected by the selected bug or are generally interested in the given
package time to pick it up from -proposed and give it a spin
themselves. Formally for us to consider an SRU to be verified requires
only one tested giving a +1 on the package, after providing test
results. The aging period (besides the obvious case of letting it run
on systems of users that always run with -proposed enabled for testing
purposes) serves the purpose of giving additional time for those
'other than the verifying person' affected users to actually test the
package and report any regressions that the original verifier could
have missed. For this, there have to be such users - and those users
need to be able to perform the validation steps as noted in the test
case.

In the case of gce-compute-image-packages, considering the nature of
the package itself, the test case, the SRU exception and the release
process of the actual package itself, I think we can be pretty
confident that the only people aware of SRU updates of those packages
are: the uploader, the SRU team and GCE upstream. These people are
probably also the only ones that can fully test the packages (since
those are usually new upstream releases), not counting cases where
there are specific isolated LP bugs involved in the upload (for those
I would say aging can still make some sense).

All those small arguments, in my opinion, sum up into a decent enough
rationale for ignoring the aging period in this case.

Cheers,

On 7 August 2018 at 22:27, Robie Basak <robie.basak at ubuntu.com> wrote:
> Hi Steve,
>
> Summary: I still think we should have an actual justification for
> waiving the aging requirement. I still don't think that we've actually
> been given a justification in this case. Give me one and I'll
> reconsider, but IMHO without any justification, we should either change
> policy to drop it wholesale, or maintain our current policy of having
> it.
>
> The rest of my response is mostly about why we have the general policy
> in the first place, rather than this specific case.
>
> On Tue, Aug 07, 2018 at 11:41:31AM -0700, Steve Langasek wrote:
>> That is certainly the purpose of the aging period, but I think this
>> overstates its value in the SRU process overall.  You say you believe you
>> have seen this happen in practice, to block a buggy SRU before it reached
>> -updates.  But broadly speaking:
>>
>>  - dist-upgrading (or trying to use update-manager) with -proposed is a bad
>>    idea for a production machine, because packages in -proposed are known
>>    not to be consistently installable throughout the SRU lifecycle
>>  - therefore, running with -proposed enabled is not something that is
>>    broadly socialized (nor should be) as a way for users to contribute to
>>    the quality of Ubuntu
>>  - therefore, random user testing coverage of SRUs is sparse, whether we
>>    wait 7 days or 30.
>
> Nevertheless, I think that wider random user testing coverage of
> packages in proposed should be something we aim to achieve. This is
> something I've mentioned before[1]. I'm confident that we have enough of
> a community willing to live on the edge that they would be willing to do
> this: if we make it easy for them and publish this as "a way to help
> out". Regardless of whether or not we can do this today, removing the
> aging period moves us in the opposite direction.
>
>> There is *some* value to an aging period, which is why I have never
>> advocated dropping the default aging period entirely.  But the value is
>> small, not straightforwardly quantifiable, and doesn't outweigh the cost in
>> human developer time of multiple round-trips with the SRU team; so whenever
>> anyone asks to waive the waiting period, and I don't have /specific/
>> concerns about the SRU, my answer will be yes.
>
> I disagree. Since aging is something we consider worthy enough to have
> in our policy by default, I think that waiving the aging period in a
> specific case should require at least _some_ justification. "I want to
> do it" or "It isn't needed" is, IMHO, not in itself a valid reason.
> Accepting those statements would in practice mean that aging doesn't
> exist in the SRU process for all those who disagree with it; in that
> case, why have an aging period in our policy at all? If we're going to
> accept that, we might as well remove the requirement entirely.
>
> On the other hand, give me an actual justification and I'd be happy to
> consider it. I just don't feel like we've been given a justification at
> all in this case, apart from "I don't like it".
>
> As for requiring an extra round-trip with the SRU team, I don't think
> this applies to SRUs in -proposed for which all the bugs are green and
> the SRU process properly followed in the SRU report by the time the
> aging period expires. In practice, I see these all being released to
> -updates promptly and without further interaction.
>
> Most delays are caused by Ubuntu developers not following documented SRU
> procedure, and it is that which causes extra round trips. That can be
> fixed by uploading Ubuntu developers easily enough; I don't think it's
> necessary to compromise quality (since we both agree that there is
> *some* value to an aging period) to do it.
>
>> I note that there's no language at all in the wiki page at
>> https://wiki.ubuntu.com/StableReleaseUpdates which talks about waiving the
>> waiting period.  I think that is an oversight; I recall seeing written
>> language at one point about this, but I can't find it now (including in the
>> history of this wiki page).  Other long-time members of the SRU team can
>> check me on this, but my understanding of the actual policy is that any
>> member of the SRU team has the authority to waive the 7-day period any time
>> they think it is appropriate to do so.  I think we should fix the wiki
>> documentation to match.
>
> My understanding matches yours. I have always understood any single SRU
> team member to have the delegated authority to waive the aging
> requirement on a case-by-case basis. What I think we're discussing here
> is when the SRU team believes they should.
>
>> > In this case, I think user-based regression alert opportunity still
>> > applies. Presumably the reason for this package to exist in the Ubuntu
>> > archive is that there are users. Removing the aging requirement removes
>> > the opportunity for those users to tell us about problems.
>>
>> The package in question, gce-compute-image-packages, is only installed in
>> cloud instances.  I believe this means that the organic regression test
>> coverage by users is LESS than it would be for another package, because the
>> test coverage we do get tends to be from individually-deployed machines (a
>> personal desktop/laptop or a one-off server), which is less likely to happen
>> in a cloud where deployments are intended to be done at scale.[1]
>
> Users managing cloud deployments done at scale following best practice
> already have the ability to test such deployments automatically,
> including packages in proposed, at virtually no extra maintenance cost.
>
> That we currently don't recommend that users enable -proposed wholesale
> is a use case that, in my view, we should fix (whether by "fixing"
> -proposed or by providing some other mechanism). I see the current
> inability as a "bug", and I don't think the existence of this bug should
> be used as a justification for removing the aging period; that would
> make it impossible to solve the problem.
>
>> And in cases where we already have an SRU exception, which dictates that
>> there must be strong upstream QA for the package before it is pushed to the
>> SRU queue, I think it's all the more reasonable to omit the default aging
>> period as part of that SRU exception.
>
> Strong upstream QA certainly does shift the balance of regression risk
> in the "safer" direction. However, packages with SRU exceptions
> generally end up with riskier stable uploads, such as feature additions.
> So that's the counterpoint that could shift the balance back and cancel
> this out. Going in the other direction, it might well be a reasonable
> mitigation for the extra regression risk in permitting wholesale
> upstream feature releases into Ubuntu stable releases to require a
> _longer_ aging period. I'm not suggesting we do this, but I hope this
> example helps illustrate my point.
>
> Robie
>
> [1] https://lists.ubuntu.com/archives/ubuntu-release/2016-October/003948.html
>
> --
> Ubuntu-release mailing list
> Ubuntu-release at lists.ubuntu.com
> Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-release
>

-- 
Łukasz 'sil2100' Zemczak
 Foundations Team
 lukasz.zemczak at canonical.com
 www.canonical.com