[SRU] OpenStack Stable Snapshots

[Brian Murray]

On Mon, Mar 08, 2021 at 11:58:18AM -0500, Corey Bryant wrote:
> On Mon, Feb 22, 2021 at 5:16 PM Corey Bryant <corey.bryant at canonical.com>
> wrote:
> 
> > Hello SRU Team,
> >
> > I'm writing to discuss delivery of patches from extended maintenance
> > stable branches as snapshots.
> >
> > This would occur during the extended maintenance phase of an upstream
> > stable branch, which falls between the maintained phase and EOL phase.
> > I would see this process being similar to the current process that is used
> > to deliver stable point releases, described at:
> > https://wiki.ubuntu.com/OpenStack/StableReleaseUpdates.
> >
> > Snapshots refers to the process by which we currently deliver package
> > updates during a development release and prior to the final release. For
> > example, nova 3:22.0.1+git2021012713.d92c0740c6-0ubuntu1 in hirsute
> > contains everything in the upstream master git branch up to commit hash
> > d92c0740c6.
> >
> > == What is Extended Maintenance? ==
> > First, it's worth understanding what the upstream "Maintained" phase is.
> > During this phase, upstream regularly releases stable point releases that
> > are based on a specific commit (e.g. the latest HEAD commit for the
> > stable/queens branch at a point in time), and we then package those point
> > releases up in distro.
> >
> > Following the "Maintained" phases is the "Extended Maintenance" phase.
> > During this phase, upstream no longer cuts any stable point releases for
> > that stable branch, however patches continue to be backported to the
> > branch. This is the phase where we would like to have the ability to
> > deliver snapshots of stable branches. This would allow us to pick up all
> > stable branch patches since, say, the final point release for stable/queens
> > (bionic). For example, the current version of nova in bionic is
> > 2:17.0.13-0ubuntu1, and that is based on the final point release for
> > upstream stable/queens. If we were to do a snapshot today for nova in
> > bionic, it would be versioned along the lines of
> > 2:17.0.13+git2021022200.944443a7b0-0ubuntu1.
> >
> > More details on upstream maintenance phases can be reviewed at:
> > https://docs.openstack.org/project-team-guide/stable-branches.html#maintenance-phases
> >
> > == Advantages of Stable Snapshots ==
> > * Less overhead for the OpenStack engineering and SRU teams. Currently we
> > have various cherry-picked patches that are dealt with via separate SRU
> > bugs requiring individual testing.
> > * As the cherry-pick process is reactive, many of these SRUs tend to be
> > high impact or critical for users, requiring immediate action from these
> > teams.
> > * We have a lot of users that are now on, and moving to, bionic (queens)
> > and we would like to be more proactive in fixing bugs for them.
> > * This would naturally result in a regular cadence along the lines of the
> > monthly stable point release cadence that we currently follow.
> > * Cherry-picking individual patches can increase regression potential for
> > frequently modified code when they don't apply cleanly.
> > * All patches that have landed in an extended maintenance branch have
> > already landed in all ensuing stable branches and master, therefore
> > negating much of the regression potential mentioned in "Disadvantages"
> > below.
> >
> > == Disadvantages ==
> > * Upstream stable branches that are in extended maintenance do not
> > guarantee the same upstream test coverage that supported branches do. The
> > maintenance-phases documentation states "There is no statement about the
> > level of testing and upgrades from Extended Maintenance are not supported
> > within the Community."
> >
> > == What Ubuntu Releases would this affect? ==
> > Stable snapshots would only be needed for Ubuntu LTS releases (Upstream
> > "Maintained" period is 18 months for stable branches). Currently this would
> > only be applicable to Ubuntu 18.04 as bionic is the only LTS with a
> > corresponding upstream stable branch that's in extended maintenance.
> >
> > == How many patches are we talking about? ==
> > For a sampling of stable/queens, new snapshots would pick up this many
> > patches:
> >
> > nova: 337
> > neutron: 106
> > heat: 47
> > cinder: 21
> > glance: 11
> > designate: 11
> > swift: 10
> > horizon: 8
> > keystone: 7
> >
> > A few of those look daunting, yes, but perhaps that depends on how you
> > look at it. Also please keep in mind, queens has been in extended
> > maintenance since October of 2019 so this would be quite the one-time
> > waterfall for bionic. In the future we'd pick up fewer patches at a time at
> > a regular cadence.
> >
> > Thanks for taking the time to read this. Please let me know if you have
> > any opinions or questions about this approach.
> >
> > Corey
> >
> 
> Hello SRU Team,
> 
> I wanted to see if I could revive this topic and discuss the possibility of
> moving forward with stable snapshots as described in this thread.
> 
> Would it make sense to add this to a meeting agenda or is the mailing list
> a good place to discuss?

The Ubuntu SRU team does have a monthly meeting (conducted via Google
Meet or whatever it is called now) and the next one is scheduled for
Thursday, March 18th. Unfortunately I won't be able to attend that
meeting but perhaps the other SRU team members can invite you so it can
be discussed.

Cheers,
--
Brian Murray