linux-headers-5.15.0-1027-gke is removed from jammy last night

Wed May 10 19:16:52 UTC 2023

Thanks for replying.
Let me clarify the use case: we use gke ubuntu containerd node image.
Because we cannot know which kernel version the node is using, on vm
creation completed, we will compile and inject a kernel module so that
pods can use that kernel module:

apt-get install -y "linux-headers-$(uname -r)" "linux-modules-extra-$(uname -r)"

> Is 5.15.0-1027-gke the version of a running kernel on your GKE instance?
yes.

>  If so, why is the headers package not installed already?
I don't think the kernel header package exists on the node image
https://cloud.google.com/kubernetes-engine/docs/concepts/node-images#ubuntu-variants.

>From that mailing list post, that timeline was not publicly announced
on gke updates, so we were completely unaware of it, and caused
incidents. Our current workaround is using 20.04 with the older gke
version because the new auto cleanup process is for jammy and later,
but I hope that older kernel can be kept around.
Does canonical collaborate gcp to create node image? If so, I can also
forward this to our gcp dedicated account team.

Thanks!

On Wed, May 10, 2023 at 8:29 AM Steve Langasek
<steve.langasek at canonical.com> wrote:
>
> On Wed, May 10, 2023 at 08:12:50AM -0700, Steve Langasek wrote:
> > When Andy and I looked at this, the analysis had showed that all our images
> > except for minimal images were being built with linux-$flavor installed,
> > rather than linux-image-$flavor, so that the headers would already be
> > present and removal of the old ABIs from the archive would have no impact on
> > users of these images.
> >
> > The GKE images are being built on a minimal base, which I did not recall.
> >
> > It is not otherwise an issue for cloud images per above, with the exception
> > of the cloud-minimal images.
> >
> > Since the linux-gke flavor is used only for minimal GKE images, we could
> > reasonably exclude these from the NBS cleaning going forward.
>
> > I've reached out to our Public Cloud team to see what their preference is.
>
> I've clarified with the Public Cloud team that, although the GKE images use
> a minimal base, the images are built using linux-gke-$version, NOT
> linux-image-gke-$version.
>
> So it's unclear that what Tai-Lin is doing is a use case that the Public
> Cloud team is concerned with supporting.
>
> Is 5.15.0-1027-gke the version of a running kernel on your GKE instance?  If
> so, why is the headers package not installed already?  If not, why are you
> trying to compile kernel modules for this version?
>
> > > Separately, whilst this NBS cleanup is in place, you can use the tool
> > > `pull-lp-debs` from ubuntu-dev-tools which should allow you to
> > > securely fetch any of the packages you require out of Launchpad
> > > Librarian archival service. (note that pull-lp-debs is part of
> > > collection of tools pull-ppa-ddebs pull-ppa-debs pull-ppa-source -
> > > which are all wrappers around the swiss army knife type of tool
> > > pull-pkg which can pull anything and everything out of Launchpad,
> > > PPAs, Debian)
> >
> >
> > > > Best,
> > > >
> > > > On Wed, May 10, 2023 at 12:51 AM Stefan Bader
> > > > <stefan.bader at canonical.com> wrote:
> > > > >
> > > > > On 09.05.23 22:28, Tai-Lin Chu wrote:
> > > > > > hi,
> > > > > > I received alerts about linux-headers-5.15.0-1027-gke being removed last night.
> > > > > > What might be the reason for that? thanks!
> > > > > >
> > > > > > Get:3 http://security.ubuntu.com/ubuntu jammy-security/restricted
> > > > > > amd64 Packages [1077 kB]
> > > > > > Get:4 http://archive.ubuntu.com/ubuntu jammy-updates InRelease [119 kB]
> > > > > > Get:5 http://archive.ubuntu.com/ubuntu jammy-backports InRelease [108 kB]
> > > > > > Get:6 http://archive.ubuntu.com/ubuntu jammy/main amd64 Packages [1792 kB]
> > > > > > Get:7 http://security.ubuntu.com/ubuntu jammy-security/main amd64
> > > > > > Packages [585 kB]
> > > > > > Get:8 http://security.ubuntu.com/ubuntu jammy-security/universe amd64
> > > > > > Packages [898 kB]
> > > > > > Get:9 http://security.ubuntu.com/ubuntu jammy-security/multiverse
> > > > > > amd64 Packages [41.2 kB]
> > > > > > Get:10 http://archive.ubuntu.com/ubuntu jammy/universe amd64 Packages [17.5 MB]
> > > > > > Get:11 http://archive.ubuntu.com/ubuntu jammy/multiverse amd64 Packages [266 kB]
> > > > > > Get:12 http://archive.ubuntu.com/ubuntu jammy/restricted amd64 Packages [164 kB]
> > > > > > Get:13 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64
> > > > > > Packages [46.6 kB]
> > > > > > Get:14 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64
> > > > > > Packages [992 kB]
> > > > > > Get:15 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64
> > > > > > Packages [1137 kB]
> > > > > > Get:16 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64
> > > > > > Packages [1143 kB]
> > > > > > Get:17 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64
> > > > > > Packages [25.6 kB]
> > > > > > Get:18 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64
> > > > > > Packages [49.4 kB]
> > > > > > Fetched 26.3 MB in 2s (10.6 MB/s)
> > > > > > Reading package lists...
> > > > > > + apt-get install -y linux-headers-5.15.0-1027-gke
> > > > > > linux-modules-extra-5.15.0-1027-gke
> > > > > > Reading package lists...
> > > > > > Building dependency tree...
> > > > > > Reading state information...
> > > > > > E: Unable to locate package linux-headers-5.15.0-1027-gke
> > > > > > E: Couldn't find any package by glob 'linux-headers-5.15.0-1027-gke'
> > > > > > E: Couldn't find any package by regex 'linux-headers-5.15.0-1027-gke'
> > > > > >
> > > > >
> > > > > That is just the normal way things change with updates. The archive will
> > > > > not find older version which have been replaced by newer ones. You
> > > > > should never try to install specific versions for that reason. Try "apt
> > > > > install linux-headers-gke".
> > > > > --
> > > > > - Stefan
> > > > >
> > > >
> > > > --
> > > > kernel-team mailing list
> > > > kernel-team at lists.ubuntu.com
> > > > https://lists.ubuntu.com/mailman/listinfo/kernel-team
> > >
> > >
> > >
> > > --
> > > okurrr,
> > >
> > > Dimitri
> > >
> >
> > --
> > Steve Langasek                   Give me a lever long enough and a Free OS
> > Debian Developer                   to set it on, and I can move the world.
> > Ubuntu Developer                                   https://www.debian.org/
> > slangasek at ubuntu.com                                     vorlon at debian.org
>
> --
> Steve Langasek                   Give me a lever long enough and a Free OS
> Debian Developer                   to set it on, and I can move the world.
> Ubuntu Developer                                   https://www.debian.org/
> slangasek at ubuntu.com                                     vorlon at debian.org