Updated release management straw man for TB consideration

[Colin Watson]

On Tue, Mar 12, 2013 at 01:33:05AM +0000, Mark Shuttleworth wrote:
> Am writing to ask you to weigh in on an updated release management
> proposal. Details are on Planet Ubuntu,

As a minor correction to your blog, raring-proposed is only intended to
be consumed by automatic-testing systems at this point; so "people who
are on the VERY BLEEDING EDGE" -> "robots who are on the VERY BLEEDING
EDGE". :-)  This is mostly expedience right now; however, I think it's a
good principle that we shouldn't waste human testing time on the same
things that are being tested automatically, but rather reserve it for
code that's passed automatic gateways.

> *1. Strengthening the LTS point releases.*
> Our end-user community will be better served by higher-quality LTS
> releases that get additional, contained update during the first two
> years of their existence (i.e. as long as they are the latest LTS).
> Updates to the LTS in each point release might include:
> 
>   * addition of newer kernels as options (not invalidating prior
>     kernels). The original LTS kernel would be supported for the full
>     duration of the LTS, interim kernels would be supported until the
>     subsequent LTS, and the next LTS kernel would be supported on the
>     prior LTS for teh length of that LTS too. The kernel team should
>     provide a more detailed updated straw man proposal to the TB along
>     these lines.

This is, I think, much like the current LTS enablement kernel scheme,
although I believe there were some adjustments that the kernel team was
contemplating.

>   * optional newer versions of major, fast-moving and important platform
>     components. For example, during the life of 12.04 LTS we are
>     providing as optional updates newer versions of OpenStack, so it is
>     always possible to deploy 12.04 LTS with the latest OpenStack in a
>     supported configuration, and upgrade to newer versions of OpenStack
>     in existing clouds without upgrading from 12.04 LTS itself.

Right.  This, I think, is also largely existing practice, though we
could do more of it, especially if we spend less effort on the non-LTS
releases.

We should think carefully about what we decide to do as optional
upgrades, though; that approach carries its own risk.  For example,
while I've been persuaded that we had little choice but to provide
separate Mesa packages to match the enablement stack rather than making
it a required upgrade, but having multiple differently-named versions of
Mesa library packages caused some interesting problems relating to
multiarch, which in particular affected Steam.  It's easier with leaf
packages, or at least things that aren't libraries.

>   * required upgrades to newer versions of platform components, as long
>     as those do not break key APIs. For example, we know that the 13.04
>     Unity is much faster than the 12.04 Unity, and it might be possible
>     and valuable to backport it as an update.

Didier commented a few days ago that we've already backported most of
the Unity performance fixes that can be safely isolated from more
intrusive work.  I don't know if his definition of "intrusive" is
mutable under this proposal, though.

> *2. Reducing the amount of release management, and duration of support,
> for interim releases*.
> Very few end users depend on 18 months support for interim releases. The
> proposal is to reduce the support for interim releases to 7 months,
> thereby providing constant support for those who stay on the latest
> interim release, or any supported LTS releases. Our working assumption
> is that the latest interim release is used by folks who will be
> involved, even if tangentially, in the making of Ubuntu, and LTS
> releases will be used by those who purely consume it.

This is the main radical change here.  We could doubtless bikeshed about
the exact value of "7" (my gut feel would be maybe a month longer, I
think), but I am in favour of this proposal.  I don't think we or our
users gain much from supporting three non-LTS releases in parallel in
the run-up to the next LTS.  We already informally taper off updates for
the oldest by means of asking uploaders if they're really sure they're
needed for something that's going to go out of support soon, but this
involves a fair amount of back and forth and is generally unsatisfactory
to me.  Relieving developers of the distraction of even having to think
about it should be a valuable reduction in their burden.

> *3. Designating the tip of development as a Rolling Release.*
> Building on current Daily Quality practices, to make the tip of the
> development release generally useful as a ‘daily driver’ for developers
> who want to track Ubuntu progress without taking significant risk with
> their primary laptop.

As you know, I'm very strongly in favour of making the development
release always usable; given the last year's progress, I think this is
an achievable goal, and the more progress we make here the easier and
cheaper it gets to make releases.

> We would ask the TB to evaluate whether it’s worth changing our
> archive naming and management conventions so that one release, say
> ‘raring’, stays the tip release so that there is no need to ‘upgrade’
> when releases are actually published. We would encourage PPA
> developers to target the edge release, so that we don’t fragment the
> ‘extras’ collection across interim releases.

Interesting suggestion.  I'm going to stew on this for a while rather
than give an immediate response :-)

My gut partial reaction is that something like this should have a new
name rather than recycling an existing one.

> As a (possibly) separate matter, in the blog I mention that decoupling
> platform and apps might help us go faster, and encourage app developers
> to make the tough choices about which versions of their apps are
> supported on which releases of the platform. I've left this bit out of
> the core proposal but would think our community would be interested in
> your collective take on that.

[This starts out as a negative response, but bear with me.]


As far as packages that are currently in the archive, I don't think it's
worth the (IMO considerable) effort to decouple much; while I recognise
that the notion is intuitively appealing to many people, the problem
tends to be that you can't actually build many newer application
packages on older releases for various reasons (and if I can figure out
the necessary code I'll see if I can get some raw data on that), and
even if you could then always doing so would make it difficult to move
forward in other areas.  This is typically due to the eternal tension
between decoupling your application from the platform and exercising
good software engineering by reusing code; if you're doing a good job of
reusing code, you're generally also making your build-dependencies
stricter, which makes it harder to decouple things.

We might be able to make some changes, but that would be traded off
against the capital cost of having to come up with some new way to share
maintenance effort with Debian (many of these applications are ones we
never touch ourselves, and during open periods the flow of changes via
syncs is generally far higher than we could sustain ourselves), or
perhaps the ongoing cost of suddenly having to maintain all these
packages directly and/or explain why we're dropping a bunch of them.
It's not clear to me that the result of all this would be particularly
better.

I don't think this kind of decoupling would help us go faster in
practice, either, as my experience of Ubuntu releases has been that
application bugs are very rarely blockers for anything much; the vast
majority of the things that we have to work hard on to get releases done
are things that we would consider "platform" in any reasonable sense
anyway.


Now, several people will no doubt tell me at this point that other
platforms manage this kind of decoupling just fine.  Yes, they do.  They
also control their entire API.  And that points to the situation where
we should be able to do this much more easily in Ubuntu: I see no reason
why apps developed using the new Ubuntu SDK, which provides a much
narrower interface and where change can be managed more carefully,
shouldn't have standard practices that permit them to be decoupled from
the platform.  I'll refrain from trying to make specific design
proposals but I'm generally in favour of them being able to update their
apps on a faster schedule and for the releases of their choice.  The
trend seems to be for that to happen in a separate archive, which seems
OK.

IOW, I would prefer to treat whatever we get from Debian plus our own
obviously-platformish additions as the "platform" here.  While this
doesn't reduce our current load, it means that we aren't faced with
trying to tag versions of some undefined number of future apps developed
with the Ubuntu phone/tablet SDK as part of each release, which I can
easily imagine as potentially slowing us down in the future; and those
new apps are by far the most tractable target for decoupling.  We
shouldn't be under the self-created illusion that this will cover
everything anyone might want to do, but the intention is already for it
to be a very substantial number.

-- 
Colin Watson                                       [cjwatson at ubuntu.com]