[PATCH 0/4] LP:1928921 LRMv5 -- switch primary version handling to kernel-versions data set

[Andy Whitcroft]

tl;dr move primary dkms-versions out of tree and consume those during package
update time.

Currently the primary dkms-versions data is committed directly to the main
kernel packages.  This allows this information to cascade reliably to all
derivatives and their associated LRM packages.  It also ensures that the entire
derivative (and dependants) tree has the same versions even if the archive
changes in the middle of the SRU cycle.  However, once the primaries are closed
it is increasingly hard to change this data.  Any partial tree update
such as is needed during an LRM-only respin requires special handling which
increases risk and complexity.

We could update the versions from the commited archive versions in every
package, but this exposes us to risk of version skew between both derivatives
and even within a dependant tree; this is particularly serious with Nvidia
where only one version is valid in each nvidia series.

To address this we propose moving the dkms-versions data into its own
repository[1].  This repository is then updated at the start of each cycle
(more often if necessary) primarily as a cycle opening task.  The current cycle
data is then pulled into the packages as they are ready to close ensuring the
builds are still independant of the data-set.

It should be noted that the dkms-versions information is not static.  It
commonly differs between SRU cycles.  When spinning (or respinning) a kernel
you need to use the version set for that cycle, as updated during and for that
cycle.  This is particularly important when respinning the previous cycle when
the current cycle has already started with updated versions.

To this end the kernel-versions repository contains branches for each cycle
(though typically these will be points in time of a linear progression of the
version information).  This allows us to select the appropriate point in the
history to extract those versions, while still allowing each of them to evolve
as needed independantly.  This model is extended to include security spins
(sYYYY.MM.DD form) which are selected from a private repository in the security
team.  Updates to dkms-versions information is primarily performed by the
`update-dkms-versions` script within the repository which understands the
vaguaries of Nvidia versioning and the preferred pockets to scan.

In order to make selection of the appropriate spin to further select the right
point in the kernel-versions history we desire the local tree to contain the
associated cycle information.  We have recently started including the spin
number in the debian*/tracking-bug file in the main kernel package.  By simply
syncing this file into the other trees as needed, we can ensure this data is
accurate in the majority case.  For security spins we will need to override it
manually.

Following this email is a patch stack to update our LRM packages to LRMv5 which
includes the new dkms-versions handling.  It consists of four patches which make
up the proposed change.  I include them here to aid review:

  UBUNTU: [Packaging] update-version -- allow specification of the master version
  UBUNTU: [Packaging] update-version -- call out to update-dkms-versions
  UBUNTU: [Packaging] update-version -- copy over tracking-bug information
  UBUNTU: [Packaging] update-dkms-versions -- update from kernel-versions

Rather than applying these manually to all of our 41 LRM packages I propose to
apply this change via an automated update squashing these into an LRMv5 update; this
mirrors the process we used for LRMv3 and LRMv4.

Proposing for all of our LRM packages.

Separatly we should include the same `update-dkms-versions` script in our main
kernels as they also include the dkms-versions data; though they only consume
the zfs and wireguard elements.  This replaces the existing
`update-versions-dkms` script as we wish to change the frequency of use of this
script and this change helps catch tooling which has not adapted.  I would
propose applying that under the same bug.

Finally we would need to modify our processes to always run
`update-dkms-versions` when present after `link-to-tracker` has run.

Comments.

-apw

[1] https://code.launchpad.net/~canonical-kernel/+git/kernel-versions

Branch: autogen5

Andy Whitcroft (4):
  UBUNTU: [Packaging] update-dkms-versions -- update from kernel-versions
  UBUNTU: [Packaging] update-version -- copy over tracking-bug information
  UBUNTU: [Packaging] update-version -- call out to update-dkms-versions
  UBUNTU: [Packaging] update-version -- allow specification of the master version

 update-dkms-versions | 150 +++++++++++++++++++++++++++++++++++++++++++
 update-version       |  43 +++++++++----
 2 files changed, 181 insertions(+), 12 deletions(-)
 create mode 100755 update-dkms-versions

-- 
2.31.1