Monthly Updates versus Monthly Images
ubuntu at kitterman.com
Tue Mar 5 23:20:14 UTC 2013
On Tuesday, March 05, 2013 06:34:29 AM Adam Conrad wrote:
> TL;DR summary: Monthly updates are harmful, monthly images are cool, let's
> do the latter without turning them into the former and all frolick happily
> in fields of time, money, and cheesecake.
> So, I'm intentionally breaking the thread here to shift gears slightly. I
> have no intention of discussing the merits of rolling versus not, of going
> on about whether now is the right time to switch, or if we need a year of
> discussion before we can consider it, or any such thing. This is purely
> on the topic of so-called "monthly updates" versus monthly images, and my
> position that the latter could prove quite useful, while the former could
> be actively harmful.
If one were going to do rolling, I mostly agree with this.
> We seem to be approaching this by conflating the two concepts of releases
> and images as if they're both the same thing and I think we can all take a
> step back and agree that, in fact, they're not. If we choose to have a
> monthly milestone planning cadence for work items and release a monthly
> image to wrap marketing and messaging around, there's definitely value in
> that. Being able to announce to the world that we had these seven major
> new features land in Xubuntu this month, these two awesome new things in
> Kubuntu, these three in Lubuntu, etc, is quite nice, and probably much less
> chaotic than having everyone plan to different cadences and sporadically
> tell the world about things as they happen. Keeping us all on a similar
> track has always proven quite helpful for interaction between teams and
> flavours, and I see significant value in that continuing.
> The above stated, I see no reason why this planning cadence and image cycle
> needs to, in any way, look like a "release", and I think it's actively
> harmful, both to our development velocity (hey look, a buzzword, you won't
> hold it against me, right?) and to our users.
> For starters, let's look at all the things I think this monthly image can't
> be, if we want to maintain a lightweight process that doesn't slow us all
> down and make small children weep:
> 1) We can't be doing mini freeze-and-release cycles before each image is
> released. I've seen it come up several times in the thread, and this
> suggestion will just slow us to a crawl. It's actively undoing any
> benefit we'd get from moving to a rolling release cycle.
We've done two day migration freezes for the relevant bits of the archive to
support Alpha 1 and Alpha 2 in this cycle. I think short migration freezes
like this (in contrast to the traditional upload freeze of 5 - 7 days) has had
almost no impact on development velocity. I've found it helpful in making
sure we're getting a good set of images for a milestone.
I think a one or two day migration freeze would be a useful thing that
wouldn't harm the goals associated with rolling in any meaningful way.
> 2) We can't be providing out-of-band critical bugfix support for these
> images. How they ship is how they are, and we need to trust that the
> preliminary smoketesting on them before they're released is enough to
> ensure an install/reboot cycle and that we won't be destroying people's
> hardware, but that's about as far as we can go.
> In short, we can't be wasting engineering time on the images themselves,
> or they become "releases", and everything that comes with that. The more
> time we spend on the images, the less time we spend on the rolling release
> itself, and the lower the quality of the LTS that comes out of the other
> end of this meat grinder.
> So, looking at various solutions proposed for this dilemma, we see people
> suggesting clever use of -updates and -security to attempt to simulate a
> monthly release, we see people suggesting phased updates dialed down to
> zero, and many other curious options. Each of these makes the assumption
> that monthly updates are a good thing. Each of these ignores the following
> facts, as far as I can tell:
> 1) If a nasty bug is discovered in the rolling release a day after we put
> out a monthly, we either need to revisit our "no SRUs to the monthlies"
> policy on a case-by-case basis, or we ask users to live with that bug
> for ~29 days until the tools say "hey, cool, you can upgrade now". In
> the current default setup, that bug would have been fixed in seven days
> or less, as that's the default update-manager nag frequency.
> 2) Security support either needs a stable base to build on (the "freeze
> the release pocket for a month" model), or it can't be provided out-of-
> band. Having the security pocket depend on things that we might need
> to remove from the release pocket is problematic.
> These two points feel like they're at odds to me, as freezing the world to
> allow an out-of-band security model means making users live with every non-
> security bug for a month until the next set of bugs are dropped on their
> heads. And, conversely, providing out-of-band security support without the
> freeze model (which has been suggested) is a bit of a nightmare from the
> archive management standpoint with, as far as I can tell, very little gain.
> All of these things in mind, I'd like to propose the following set of ideas,
> and see if we can massage them into something palatable to everyone's
> 1) No freezes, no pre-release process, monthly images are merely a fully
> smoketested daily being released on a predetermined date, no better or
> worse than the previous or next smoketested daily, just what happens
> to be in the archive on that date. We target our work items to that
> date, and if they slip, so be it, the features that make it in can then
> be mentioned in messaging for 13.05, 13.06, etc, those that don't can
> be postponed for the next month.
I'd suggest a migration freeze from when the first image is built until testing
is done. These do need to actually support doing installs, not eat hardware,
etc. and so I think holding migrations to make it possible to address critical
issues (and mean critical), respin, and move on will be useful and minimally
> 2) No out-of-band support at all, SRU or security. The only slight change
> from how we do things now would be that security updates destined for
> the development release would be built in the security PPA (which does
> not build against -proposed), so they don't pick up new dependencies
> and can then be copied to the archive and not accidentally get caught
> up in library transition snags that hinder their migration to the
> release pocket.
If this is going to be messaged as anything other than the development series,
I think that rolling needs to be included in USNs. I don't think it's
appropriate to leave rolling out and leave users with the impression it isn't
getting security fixes when we've told them it's ~safe to use.
> 3) We twiddle the phased-updates spec a teensy bit so that P-U-P values
> over 100 are treated by update-manager as security/critical updates,
> and offered immediately, rather than after the configured update delay,
> much as packages in the -security pockets are now offered. With this
> model, we can make the scripts that copy from the security PPA to the
> archive set the phased update probability to 101 for security uploads,
> and have them treated as "special" by update manager, without having to
> actually use the -security pocket and deal with the annoyance of a
> pocket that doesn't have a stable base to depend on.
> 4) We leave the update-manager default at seven days (which I feel is a
> perfectly sane compromise between "getting bugfixes out quickly" and
> "not bugging users constantly or eating their bandwidth"), but add an
> option for "every four weeks", for people who really feel that the
> current top-end value of two weeks is still too frequent for them.
> Again, I can't stress enough that I feel that artificially delaying updates
> to users to create the illusion that we're "releasing" once a month is not
> doing anyone any favours, not us or them. We'll be getting bug reports on
> "ancient" versions, and constantly asking people to muck with whatever
> config is necessary to get off the monthly track and on to the rolling one
> just to get their fix, we'll be asking users to run software that none of
> us will be running, because we all want/need to have up-to-date systems to
> develop against, and we'd have no one but ourselves to blame for this state
> of affairs, as we're the ones who created the belief that updates only
> happen once a month, and that bugs are just something you live with until
> the next magical code drop four weeks away.
> If we were going to freeze every month to stabilize, and offer proper SRU
> bugfixes to those mini releases, I'd be all for monthly updates, except for
> the part where our community, both paid and unpaid, would need to be about
> twice as large to accomodate such lofty goals. As it stands, I see no way
> that we can offer a monthly release without shafting both ourselves and our
> This may well read as a disjointed and potentially grumpy view of the world,
> and I imagine such an interpretation wouldn't be far off, but I feel quite
> strongly that the idea of withholding updates from users isn't just a poor
> decision but is, in fact, actively harmful. I know I have colleagues who
> agree with me on that point, and I've seen a few speak up in the previous
> With that in mind, I'd like to ask that conversations around monthly updates
> shift from being "how do we provide monthly updates" to "how do we meet our
> needs without providing monthly updates". The two greatest needs I see are
> security support and giving users the option for lower frequency of less
> critical updates. I think leveraging phased-updates to that end is a
> fairly simple, lightweight, and elegant solution that completely avoids the
> harm and complexity of any of the proposed monthly updates schemes.
I think for phased updates to be meaningful in this context it would have to
be a bit different. As (IIRC) Highvoltage mentioned, he wants to be at the
back of the iceberg:
What's needed is not only show me updates every X days, but only show me non-
critical updates that are at least X days old. I don't think we have a way to
More information about the ubuntu-devel