Follow Up from "Let's Discuss Interim Releases"

[Robert Collins]

On 8 March 2013 14:19, Rick Spencer <rick.spencer at canonical.com> wrote:
> Hi all,
>
> There has been a lot of discussion and impact around the strawman
> proposal for changing our release cadence that I sent last Thursday.
> There was a misconception that the proposal was a decision that I was
> masking as a call for discussion. I want to reassure everyone that I
> really did mean it as a discussion. I feel passionately that we need
> to change and innovate in this area, but a change like this cannot
> succeed, or in fact be made, without discussion in the community and
> proper governance.

FWIW it was clearly a proposal, not a fait accompli being announced,
at least for me :).

> Discussion of this topic on the mailing list and at UDS this week was
> wide ranging. There were a lot of divergent opinions and ideas. The
> discussion seems to have resulted in roughly three different forms of
> proposals.
>
> 1. Move to a rolling release similar to what I proposed in the
> original straw man.
> 2. Continue to release interim releases but only support them until
> roughly the next interim release 6 months later.
> 3. Dramatically increase the rate of our releases to, say, once per month.
>
> I've attempted to capture the essence of these proposals (and
> associated sub-proposals) along with a structure for points and
> counterpoints in wiki format to support honing and organizing. They
> are currently stubs, so will need detailed content and continued
> honing, but the wiki format invites collaboration on that honing.
>
> See:
> https://wiki.ubuntu.com/ReleaseCadence
> https://wiki.ubuntu.com/ReleaseCadence/RollingRelease
> https://wiki.ubuntu.com/ReleaseCadence/SixMonthInterimRelease
> https://wiki.ubuntu.com/ReleaseCadence/TrueMonthlyReleases
>
> I'd like to invite everyone who is interested to get their input into
> these pages by March 18th (or thereabouts). Then I'd like to work with
> interested people to select what we consider the best proposal to take
> to the technical board for guidance.
>
> Part of the straw man proposal was to convert 13.04 into a Rolling
> Release in order to allow us to go faster on the converged OS starting
> immediately. Given the work that is left to achieve a proper proposal
> for the tech board, I don't foresee such a proposal being completed in
> time to make such a radical change to 13.04.

One thing I think has been missing from the discussion so far has been
in reviewing our *definition* of a release. There was some mention of
defining what Ubuntu is - how big the surface area we maintain, vs app
developers maintain and deliver /on/.

I'd like to decompose 'Release' though. Technically, it means a number
of things today:
A - an archive that we place a high friction change process on,
intended to eliminate regressions [the SRU]
B - a logical name that users can associate with a /large/ bundle of
changes. One can say 'Unity in Raring is fantastic' only because
Raring is a thing.
C - A set of APIs that work well together and which we commit to keep
working for the duration of the release (except where it is infeasible
[e.g. due to facebook dropping their support for the server side of an
API].
D - A mostly unchanging environment. No surprises.

We've got good reasons to want to achieve those four things:
A - We don't want millions of users to have something that worked
break because we changed something badly. Breaking folks environments
is a good way to encourage them to move elsewhere.

B - We have 20K pieces of software, 2K in main, all changing all the
time.Talking about individual versions of everything is tedious, for
all that it is accurate.

C - We want to get app developers vendors - both individuals and
aggregators like steam - to target Ubuntu, so that more folk can
benefit from a free and open platform. I broadly model app vendors as
minimaxers (for both pro bono and commercial vendors): for the least
investment, get the most gain from from building and supporting an app
on a platform. Each new platform needs a minimum reach to be worth
porting / rewriting on, and the harder it is to build the app there,
the larger the reach required for it to be worth it.
Crucially, the faster APIs change, the wider the reach needed by the
platform for supporting it to be worth it. (Revenue = total users *
per-user-revenue, *or* good-done = total users * benefit per user) /
effort.
APIs don't need to be unchanging, they just need to be
unchanging-enough that working with them is worth it.

D - Since we are optimised for producing releases, we want to minimise
the cost of supporting each release: the more it changes, the more we
(and others) have to revise documentation, screenshots, manuals,
wrapper scripts, and the more we run the risk of causing a regression.

Now, I think it is axiomatic that we need to keep achieving the four things:
 - reliability for our users.
 - clarity of communication about what upstream software is available
and its versions
 - good APIs that work together well and are stable enough to be
attractive for app vendors
 - no surprises for our users / OEMs / app vendors

However, are releases the best way to do this - at all? Currently, we
/don't/ achieve all four things. We've changed the APIs we recommend
app developers use multiple times - not tweaks, wholesale replacement.
We are mostly reliable within a release, but regressions aren't
unknown. We force a 6 month refresh rate on all our docs and APIs,
because we don't make commitments across releases [and the whole 'we
get most from upstream makes that a complex thing to reason about].
[The retooling thing for OEMs every 2 years seems like a fairly
significant fixed overhead to be aware of in any proposal - which is
perhaps why LTS's have always been on the table in this discussion
:)].

So, here is an alternative approach to solving these four things while
still integrating upstream changes etc.

tl;dr version:
 - Identify a really small set of APIs to support, and set LTS
timeframes on them. Examples - boot configuration, networking,
graphics, etc.
 - For new APIs set timeframes that reflect their maturity (and
support them for that timeframe).
 - Support those APIs for that timeframe irrespective of upstream -
but in selecting them discuss that with upstream.
 - Use feature flags to decouple 'new code release' from 'new
behaviour' for the platform itself - Unity, indicators etc.
 - For key upstream applications (e.g. using data from popcon) make
sure we allow co-installability of multiple versions. (libreoffice may
be an example of this).
 - Move the concept of using 'a release of Ubuntu' to using 'a
configuration' - LTS is 'keep my behaviour unchanged', interim
releases are 'give me new features when they are production quality'
and 'development edition' is 'give me new stuff as soon as it enters
the archive'.
 - Stop doing fixed releases altogether.

The engineering around this seems fairly straightforward to me: the
phased updates work + CI tooling can combine quite nicely into a CD
story.

I'm happy to provide a longer version and engage on implementation
details, if this sounds like an interesting approach.

AFAICT it should meet all the audiences we have, though it is quite
visibly different to the way we deliver it today.

-Rob
-- 
Robert Collins <rbtcollins at hp.com>
Distinguished Technologist
HP Cloud Services