Re-evaluating Ubuntu's Milestones

Simon Quigley tsimonq2 at ubuntu.com
Mon Apr 9 01:46:11 UTC 2018


Hello,

This past cycle was interesting, with Spectre/Meltdown complications
among other things causing the first two Alphas to be cancelled, and on
the Lubuntu side of things, we missed Beta 1 because of a critical bug
that wasn't noticed due to lack of testing. This cycle made me question
the purpose and usefulness of the milestones in a release cycle.

If we do e.g. Alpha 1 as a community, these images are frozen in time
and are said to be "safe to install" when in reality, further bugfixes
can be spun in the daily ISO less than 24 hours later, making that
milestone and the several days leading up to it something which will
just be paved over the next day. In reality, due to the Proposed
Migration procedures we have set in place[1], the release pocket should
*always* be installable and working. Effort could be better spent making
sure that the tests for these packages are better and that continuous
integration is improved all around, rather than just hard freezing the
archive and doing manual tests for some flavors, which often times
blocks the work of others.

I wanted to see if my thoughts were shared, and after speaking to
several people from different flavors (albeit fairly informally, so
these should not be treated as final opinions, but from Xubuntu, Ubuntu
MATE, Kubuntu, and Ubuntu Budgie) regarding the state of milestones in
the archive, it seemed like we shared common points, and it was clear to
me that it would be a good idea to formally propose some change in these
procedures.

I am proposing that we get rid of the Alpha and Beta 1 milestones
entirely, and we organize a monthly testing "week" (Tuesday through
Thursday), which involves no archive freezes, no formally released ISOs,
but testing of every product (community and Canonical) that is typically
shipped as part of a final release (so including Desktop and Server).
The goal would be to ensure that any issues are ironed out early on, to
foster collaboration between people (Canonical, community, and everyone
in between) regarding what's working and what isn't thus far in the
cycle, and plans for the rest of the cycle (possibly via a short mailing
list thread, something on the community hub, or whatever happens to work).

So a (tentative) schedule would look like this for 18.10:
 1. At the beginning of the cycle, the usual toolchain uploads are done,
the autosyncer is turned on, and the "floodgates" are opened. This could
potentially take us to the end of May, depending on how many transitions
are started, what kind of changes are coming in from Debian, how quickly
people can get things organized/migrated, and when Mark decides to
announce the Calculating Camel. If the archive is consistent enough by
the last weekend of May (26th/27th), we could do an organized testing
session from then (so, the 29th through the 31st). This makes sure that
once all of the transitions are finalized "for now" that nothing broke
between the Bionic DIF[2] and that point.
 2. By the latter half of June, we have hit the Feat. Def. Freeze[3] for
some packages in Main, and the transitions which started from the
floodgates being opened should be calmed down. The last week of June
(26th through the 28th) could be another coordinated testing period.
 3. In comes July, which is really the last full month you can land
features that should be included in the 18.10 release, according to the
usual timing of the release cycle. We would do coordinated testing that
last week as well (24th through the 26th). Any last transitions could be
discussed which affect everyone and could be landed in time for Feature
Freeze.
 4. Feature Freeze comes in on the week of August 23rd, ish. That would
put coordinated testing right after that (28th through the 30th), and
makes it important for there to be testing, because we can assume that
features are no longer the primary focus of the flavors for the upcoming
release. If upstream release cycles for common components happen to line
up in a way which makes it desirable to ship in a release, that could be
discussed. Otherwise, deep testing of edge cases (unusual install
methods come to mind right away) would become a more specific goal, so
we can ensure that the final release is as bug-free as possible.
 5. In between that and the latter end of September would be UIF[4] and
the Documentation String Freeze[5]. Additionally, this is also the
typical spot of Final Beta, and the goal (since we didn't release a
"Beta 1") would be to release an image, but call it "18.10 Beta" (thus
removing versioned betas and just declaring it to be a "beta"). The
archive would freeze as normal, Beta images would be released, and the
archive would enter the usual semi-frozen state where seeded packages
need manual approval. From this point on, the release cycle would
proceed as it always has.

Thoughts?

[1] https://wiki.ubuntu.com/ProposedMigration
[2] https://wiki.ubuntu.com/DebianImportFreeze
[3] https://wiki.ubuntu.com/FeatureDefinitionFreeze
[4] https://wiki.ubuntu.com/UserInterfaceFreeze
[5] https://wiki.ubuntu.com/DocumentationStringFreeze

-- 
Simon Quigley
tsimonq2 at ubuntu.com
tsimonq2 on freenode and OFTC
5C7A BEA2 0F86 3045 9CC8
C8B5 E27F 2CF8 458C 2FA4

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: OpenPGP digital signature
URL: <https://lists.ubuntu.com/archives/ubuntu-release/attachments/20180408/d043e663/attachment.sig>


More information about the Ubuntu-release mailing list