Re-evaluating Ubuntu's Milestones

[Steve Langasek]

On Mon, Apr 09, 2018 at 02:31:31PM -0500, Simon Quigley wrote:
> On 2018-04-09 03:30, Oliver Grawert wrote:
> > well, apart from actual installer fixes, your users should get all
> > these fixes through package updates anyway ... 

> Right, which is another point for getting rid of these extra milestones, in
> my opinion.

> > One thing that the other pro/con responses did not cover yet but that
> > should not be underestimated is the promotional aspect of milestones
> > ...  

> > You typically get press coverage for such pre-releases and will likely
> > attract more testers.

> Not really, actually. In my experience, testers are present when there is an
> occasion to test them, regardless of putting a name to it or releasing an
> ISO. I could see your point if my proposal was to get rid of the milestones
> entirely with no replacements, but in this case, having the testing week
> once a month would attract press coverage as well.

> Why? Because milestones in all reality are just a fancy name to slap on an
> ISO that will most likely be stale the next day. You then get people
> installing from these ISOs and potentially even reporting old bugs. The
> unfortunate reality of this press coverage is that you could pick an ISO any
> day of the month and call it "beta," and just because it has that name on it
> means that people will install it because of the appeal of the name. Despite
> the positive press that comes from the associated announcements (that can
> always be made regardless, which is what Lubuntu has started doing[1]), in a
> technical sense, I would even consider it *bad* for people to install using
> these ISOs.

> The coordinated testing weeks would allow for there still to be positive
> press coverage (and maybe announcements resulting from cross-team
> collaboration during these times) while not having the downsides of a
> blessed image when the archive isn't in a decently stable state.

I agree (as you know).

The one other value milestone images provide is to give a "known good" image
to install the development release from.  We have solved this for Ubuntu
Desktop and Server by having automated tests that gate the promotion of an
image build to "current".  I would strongly encourage flavors to collaborate
around this automation, instead of continuing to rely on a heavy-weight
manual test process that leaves the "known good" image stale for weeks at a
time.

Code for this automation lives here:

  https://code.launchpad.net/~ubuntu-test-case-dev/ubuntu-test-cases/desktop

If there is interest from flavors in having this image gate, I would be
willing to argue for Canonical hosting of the test infrastructure, as
necessary.

And if there *isn't* interest from flavors in doing this, well, I also don't
think that should block on that as a reason to carry on with the existing
milestone process, which I think is a very inefficient use of everyone's
time.

So to summarize, I think the right path forward is:

 - discontinue all opt-in milestones for 18.10 and beyond
   - implicitly discontinuing the matching milestone freezes
 - coordinate a cadence of "testing weeks", organized by the flavor leads
   (i.e.: requires no involvement from ubuntu-cdimage or ubuntu-release in
   order to drive to success)
 - at the flavor teams' discretion, implement automated QA gating of daily
   image promotions

-- 
Steve Langasek                   Give me a lever long enough and a Free OS
Debian Developer                   to set it on, and I can move the world.
Ubuntu Developer                                    http://www.debian.org/
slangasek at ubuntu.com                                     vorlon at debian.org
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: not available
URL: <https://lists.ubuntu.com/archives/ubuntu-release/attachments/20180409/a2c368b8/attachment.sig>