Release management strategy for Alpha releases ("Tribes")

[Matt Zimmerman]

On Mon, Aug 13, 2007 at 12:47:27PM +0200, Martin Pitt wrote:
> I think that the most basic question that we have to answer ourselves
> is: "What do we expect from a Tribe release?" 

I'll refer to "Alpha" releases below, to be more general.  Our documentation
should do the same, and I'd like to propose that starting with Gutsy+1, we
use that terminology in all our communications about the CDs as well.

> Small approach: We can regard it as a mere snapshot in development,
> with barely enough effort to make it installable. But in this case we
> should not advertise them so heavily (4) and e. g. only write about
> new features on the Beta release notes and web pages. The main focus
> would be to exercise the installer and test the kernel on a broad
> selection of hardware.

Early alphas can't be much more than this, and should be characterized as
such.  They're snapshots of a fast-moving target, and will have known
problems which aren't important to fix at this stage.

I don't agree that we should be less communicative about them for this
reason.  They will contain new kernels, new X drivers, and a variety of
other bits and pieces which need to be tested, and tested early.  Much of
this can be accomplished simply by booting the live CD and providing
feedback, even if the user doesn't install it.  Proper testing of these
changes requires broad participation, and we only have a chance at that when
lots of people hear about it.

> Big approach: We regard them as a mini-release which we want both
> developers and advanced users to test, where new features actually
> work properly and which we can be proud of. We would focus on
> presenting them well instead of presenting them as fast as possible.
> Then we must concentrate on bug fixing and radically reject new
> features which have not matured enough so at least work for ourselves.

Later alphas are more like this, and folks can expect to install them if
they plan to track the development branch, etc.  Their primary purpose is
still testing, though, more so than showcasing features.  It's not until the
beta release that we should promote these features for general consumption
and evaluation (as opposed to testing).

Given the change in goals, perhaps it would make sense to name these two
phases differently?  Snapshot and alpha?  Pre-alpha and alpha?

It's also a bit confusing that we continue to name the milestones the same
way after the beta release.  Perhaps those should start with "beta 2".

> With snapshots, we should IMHO drop the strict schedule and instead
> release tribes "when they are ready".  We do a relatively liberal
> freeze, and when a daily image is relatively consistent, we send it
> out. This would both remove pressure on the developer team and release
> management and would also calm down the pre-freeze rush to break the
> distro on several places at once.

I think this is fine, with one caveat: the snapshots *must* go out.  This is
an important testing vehicle and also keeps the pace of activity visible in
the community.  The exact schedule is not very important, and doesn't need
to be predictable, but it absolutely must happen.

This is pretty much what we did with earlier releases, and the problem we
encountered was that, in the absence of a set date, milestones just weren't
a priority, and so we went many weeks with relatively little community
testing, only to discover important bugs later on.

> With mini-releases we need to extend the freeze by several days (7
> IMHO) so that the pre-release feature rush happens earlier, we have
> a realistic chance of unbreaking them and giving them some polish, and
> can postpone new features to the next Tribe. This obviously means a
> lot of developer and RM time overhead, and thus we need to lower the
> frequency of Tribes. In this model, three weeks are ambitious and two
> weeks are insane, so we need to drop the idea of the 14-day interval 
> before UVF and FF.

Freezes aren't the only way to communicate with developers; if what we need
is for major changes to be deferred until after the milestone, we could ask
for that without imposing a freeze (which creates more work for everyone).

A random technical idea I've considered is to add to dput/dupload a check
for the current release guidelines, displaying them to the user to get
confirmation that their upload is appropriate for the present time.

> If we aim for the latter approach, we should also modify our current
> approach of one huge "break it first, fix it later" development curve
> to a series of smaller ones which can be controlled more easily, and
> therefore move the feature freeze to a much later point of the release
> cycle.

There are currently only 4-5 weeks between Feature Freeze and the beta
release, and a comparatively short beta period as well.  I don't think that
moving feature freeze much later would be feasible without extending the
release cycle.

That said, I support more continuous fixing and development, rather than one
big cycle.  I think it's necessary to accept a high degree of breakage early
in the release cycle, regardless of this, though.  We could define this
period better, though; in practice, it seems to be considered acceptable for
things to be broken until feature freeze, which leaves relatively little
time for dedicated fixing.

-- 
 - mdz