[PATCH 0/6][Unstable] Kernel package builds running out of space on builders

[Andy Whitcroft]

After some discussion and further testing we have ascertained that the
fixperms phase is already being handled by `dh_fixperms` so that patch is
not required.  With that dropped:

Acked-by: Andy Whitcroft: <apw at canonical.com>

-apw

On Thu, 3 Jun 2021 at 14:05, Seth Forshee <seth.forshee at canonical.com>
wrote:

> BugLink: https://bugs.launchpad.net/bugs/1930713
>
> [Impact]
>
> Our kernel builds are sometimes running out of space on the builders when
> we
> are building multiple flavours. We've seen this with focal:linux-hwe for
> amd64
> and impish:linux-unstable for arm64. This is in part because package
> builds are
> broken up into a build phase (which builds the source tree) and a binary
> phase
> (which creates the debs). These are run separately, with the binary phase
> run
> under fakeroot to get correct ownership for files in the package archives,
> requiring builds for multiple flavours to be present on disk at the same
> time.
>
> We have implemented various fixes for this problem over time, and explored
> many
> others which have not worked out. But the size of the kernel keeps
> increasing,
> and now it seems our only remaining option is to build one flavour and
> install
> its files, then remove the flavour build files before building the next
> flavour. This means that files are installed for later package builds
> during
> the build phase, requiring that ownership of these files be fixed up
> during the
> binary phase to get correct ownership in the package archives.
>
> [Test Plan]
>
> Build a full set of kernel packages (inlcuding linux-source and dbgsym
> packages, which are generally excluded when not building on builders) at a
> given tag, then build another set from the same tag with the packaging
> changes
> applied. Compare the resulting debs to confirm that the set of produced
> packages is the same, the file lists within the packages are the same, and
> that
> file ownership and permissions between the packages is identical.
>
> I have done this testing with the proposed patches with a recent
> linux-unstable
> tag and found no differences with and without the changes.
>
> [Where problems could occur]
>
> Incorrect ownership of files in the package archives is the main concern. I
> have tested for this, but it is possible that future upstream changes could
> unexpectedly result in files with incorrect ownership.
>
> Reordering of the package build sequence could result in missing files
> which
> should be in packages, or files present in packages which should be
> excluded.
> Some instances of this occurred while developing these changes and have
> been
> fixed. Future updates to upstream or to the packaging could cause
> additional
> issues.
>
> Thanks,
> Seth
>
> ---
>
> Seth Forshee (6):
>   UBUNTU: [Debian] use stamps for flavour install targets
>   UBUNTU: [Debian] run install-$(flavour) targets during build phase
>   UBUNTU: [Debian] remove dh_testroot from install targets
>   UBUNTU: [Debian] dkms-build -- use fakeroot if not running as root
>   UBUNTU: [Debian] exclude $(DEBIAN)/__abi.current from linux-source
>   UBUNTU: [Debian] fix ownership for files installed during build
>
>  debian/rules.d/2-binary-arch.mk  | 88 ++++++++++++++++++++++----------
>  debian/rules.d/3-binary-indep.mk | 18 ++++---
>  debian/rules.d/4-checks.mk       |  6 +--
>  debian/scripts/dkms-build        |  4 +-
>  4 files changed, 78 insertions(+), 38 deletions(-)
>
> --
> kernel-team mailing list
> kernel-team at lists.ubuntu.com
> https://lists.ubuntu.com/mailman/listinfo/kernel-team
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.ubuntu.com/archives/kernel-team/attachments/20210604/2d34ada2/attachment.html>