Proposed pocket racing uninstallability and SRU verification around release time

[Steve Langasek]

On Wed, Oct 19, 2016 at 09:19:50PM +0100, Robie Basak wrote:
> On Wed, Oct 19, 2016 at 12:41:26PM -0700, Steve Langasek wrote:
> > I understand the intent.  Testing -proposed as a whole then still leaves you
> > the problem of root-causing any regressions.  Have you had success in
> > feeding back the results of your testing into the SRU process?

> I'd like to present another use case. In the "devops" world, it is
> considered best practice to have tests for automated deployments. This
> means that these users have CI - they can see at a glance whether
> everything they need in their deployment still works.

> For users who do this, it would be trivial to add -proposed to their
> matrix. Root causing on a red would still be necessary, but they would
> need to do this anyway if they don't tell us about a regression and
> their deployments from -updates goes red.

> I don't currently know of anyone actually doing this for -proposed, so
> perhaps it isn't worth the effort to fix the other issues you point out
> that currently stop testing -proposed as a whole being useful in this
> way. Perhaps we can revisit this (for this particular use case) if users
> actually tell us they want this.

FWIW my view here is that -proposed would always be too noisy to be useful
in this way.  We don't run our own archive-level CI (i.e. autopkgtest) this
way; instead, we cherry-pick individual SRUs from -proposed and regression
test them vs. the existing "trunk" archive, which is release+updates.  I
would not expect a downstream consumer of these to want to try to stay on
top of failures introduced by packages that haven't passed our own upstream
CI gate yet.  You're certainly not going to deploy the lot from -proposed
into production (if you're sane), so why track it with CI?

If you're not committed to keeping the CI green, it's just a waste of
cycles.  And if it's not in the pipeline to be deployed, you're not actually
committed to keeping it green.  So just run the CI for the thing you really
care about the answer to.


> > AIUI Robie's concern was that -proposed is a mess immediately after a
> > stable release.  And it was *my* argument that this is a quantitative,
> > rather than qualitative, difference from the rest of the cycle.  :)

> Point taken, thanks. We should document that enabling proposed wholesale
> is not recommended, as I think the documentation currently is the
> opposite of this. It should probably be removed from the GUI option that
> can enable it then, too, since to my knowledge that doesn't enable any
> kind of apt pinning to stop a wholesale update to proposed.

> IOW, the top part of https://wiki.ubuntu.com/Testing/EnableProposed is
> broken. "Selective upgrading from -proposed" (the second section) should
> be the default, and the GUI shouldn't lead people into doing the first
> but not the second.

> Should we perhaps ship a proposed pin in preferences.d by default?

Yes, this is what I would like to see happen.  But somehow, despite this
having been discussed before, there's been a reluctance to implement it.  I
don't remember there being any problematic side effects to us making this
change; perhaps someone else on the list does.

The only complication I'm aware of is that we absolutely must have different
apt preferences behavior for chroots - such as buildd chroots - than for end
user systems; but that should be solvable by using a preferences.d file
shipped in the right package.  Probably either software-properties-common or
ubuntu-release-upgrader-core?

-- 
Steve Langasek                   Give me a lever long enough and a Free OS
Debian Developer                   to set it on, and I can move the world.
Ubuntu Developer                                    http://www.debian.org/
slangasek at ubuntu.com                                     vorlon at debian.org
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: not available
URL: <https://lists.ubuntu.com/archives/ubuntu-release/attachments/20161019/8e6b21e3/attachment.pgp>