[PULL] [L/unstable] merge configs into annotations
Andrea Righi
andrea.righi at canonical.com
Wed Nov 16 18:08:23 UTC 2022
[Overview]
Each Ubuntu kernel needs to maintain its own .config for each supported
architecture and each flavour.
Every time a new patch is applied or a kernel is rebased on top of a new
one, we need to update the .config's accordingly (config options can be
added, removed and also renamed).
So, we need to make sure that some critical config options are always
matching the desired value in order to have a functional kernel.
[State of the art]
At the moment configs are maintained as a set of Kconfig chunks (inside
debian.<kernel>/config/): a global one, plus per-arch / per-flavour
chunks.
In addition to that, we need to maintain also a file called
'annotations'; the purpose of this file is to make sure that some
critical config options are not silently removed or changed when the
real .config is re-generated (for example after a rebase or after
applying a new set of patches).
The main problem with this approach is that, often, we have duplicate
information that is stored both in the Kconfig chunks *and* in the
annotations files and, at the same time, the whole .config's information
is distributed between Kconfig chunks and annotations, making it hard to
maintain, review and manage in general.
[Proposed solution]
The proposed solution is to store all the config information into the
"annotations" format and get rid of the config chunks (basically the
real .config's can be produced "compiling" annotations).
[Implementation]
To help the management of the annotations an helper script is provided
(debian/scripts/misc/annotations):
```
usage: annotations [-h] [--version] [--file FILE] [--arch ARCH] [--flavour FLAVOUR] [--config CONFIG] (--query | --export | --import FILE | --check FILE)
Manage Ubuntu kernel .config and annotations
options:
-h, --help show this help message and exit
--version, -v show program's version number and exit
--file FILE, -f FILE Pass annotations or .config file to be parsed
--arch ARCH, -a ARCH Select architecture
--flavour FLAVOUR, -l FLAVOUR
Select flavour (default is "generic")
--config CONFIG, -c CONFIG
Select a specific config option
Action:
--query, -q Query annotations
--export, -e Convert annotations to .config format
--import FILE, -i FILE
Import a .config into annotations
--check FILE, -k FILE
Validate kernel .config with annotations
```
This script allows to query config settings (per arch/flavour/config),
export them into the Kconfig format (generating the real .config files)
and check if the final .config matches the rules defined in the
annotations.
Examples (annotations is defined as an alias to
debian/scripts/annotations):
- Show settings for `CONFIG_DEBUG_INFO_BTF` for master kernel across
all the supported architectures and flavours:
$ annotations --query --config CONFIG_DEBUG_INFO_BTF
{
"policy": {
"amd64": "y",
"arm64": "y",
"armhf": "n",
"ppc64el": "y",
"riscv64": "y",
"s390x": "y"
},
"note": "'Needs newer pahole for armhf'"
}
- Dump kernel .config for arm64 and flavour generic-64k:
$ annotations --arch arm64 --flavour generic-64k --export
CONFIG_DEBUG_FS=y
CONFIG_DEBUG_KERNEL=y
CONFIG_COMPAT=y
...
- Update annotations file with a new kernel .config for amd64 flavour
generic:
$ annotations --arch amd64 --flavour generic --import build/.config
Moreover, two additional kernelconfig commands are provided
(debian/rules targets):
- listnewconfigs: allow to generate a list of new config options (e.g.,
after a rebase) and store them in CONFIGS/new-<arch>-<flavour> for
review
- importconfigs: after new .config's are generated and reviewed (in
CONFIGS/<arch>-<flavour>) we can use this command to automatically
import all of them into the local annotations
[Pros and Cons]
Pros:
- avoid duplicate information in .config's and annotations
- allow to easily define groups of config settings (for a specific
environment or feature, such as annotations.clouds, annotations.ubuntu,
annotations.snapd, etc.)
- config options are more accessible, easy to change and review
- we can easily document how config options are managed (and external
contributors won't be discouraged anymore when they need to to change a
config option)
Cons:
- potential regressions: the new tool/scripts can have potential bugs,
so we could experience regressions due to some missed config changes
- kernel team need to understand the new process (even if everything
is transparent, kernel cranking process is the same, there might be
corner cases that need to be addressed and resolved manually)
[TODO]
- Automatically migrated old config chunks into annotations (this is
easy, but we need to document the procedure or provide a script to do
it automatically)
- Migrate flavour and arch definitions into annotations (rather than
having this information defined in multiple places inside
debian/scripts); right now this information is "partially" migrated,
meaning that we need to define arches and flavours in the headers
section of annotations (so that the annotations tool can figure out
the list of supported arches and flavours), but arches and flavours
are still defined elsewhere, ideally we would like to have arches and
flavours defined only in one place: annotations.
---
The following changes since commit 1938fabc4cf0d4ed450f4cb2b02cfd48b5e4f957:
UBUNTU: Ubuntu-unstable-6.1.0-5.5 (2022-11-07 16:44:48 +0100)
are available in the Git repository at:
git://git.launchpad.net/~arighi/+git/lunar annotations
for you to fetch changes up to cc5ca9b85d730f01bdbde450cd0141405a2ee082:
UBUNTU: [Config] drop configs and rely on annotations (2022-11-16 17:28:47 +0100)
----------------------------------------------------------------
Andrea Righi (13):
UBUNTU: [Packaging] config-check: ignore values that are not defined in annotations
UBUNTU: [Packaging] config-check: do not strictly enforce CONFIG_CC_VERSION_TEXT
UBUNTU: [Packaging] introduce annotations script
UBUNTU: [Packaging] automatically generate configs from annotations
UBUNTU: [Packaging] drop deprecated script splitconfig.pl
UBUNTU: [Packaging] drop deprecated script tristate.sh
UBUNTU: [Packaging] drop config-check and use annotations
UBUNTU: [Packaging] simplify kernelconfig
UBUNTU: [Packaging] kernelconfig: always keep configs
UBUNTU: [Packaging] provide listnewconfigs
UBUNTU: [Packaging] kernelconfig: introduce importconfigs
UBUNTU: [Config] import all configs into annotations
UBUNTU: [Config] drop configs and rely on annotations
debian.master/config/amd64/config.common.amd64 | 701 -
debian.master/config/amd64/config.flavour.generic | 3 -
debian.master/config/annotations | 6808 +++++-----
debian.master/config/arm64/config.common.arm64 | 731 -
debian.master/config/arm64/config.flavour.generic | 14 -
.../config/arm64/config.flavour.generic-64k | 14 -
debian.master/config/armhf/config.common.armhf | 712 -
debian.master/config/armhf/config.flavour.generic | 15 -
.../config/armhf/config.flavour.generic-lpae | 15 -
debian.master/config/config.common.ubuntu | 13418 -------------------
debian.master/config/ppc64el/config.common.ppc64el | 702 -
.../config/ppc64el/config.flavour.generic | 3 -
debian.master/config/riscv64/config.common.riscv64 | 694 -
.../config/riscv64/config.flavour.generic | 3 -
debian.master/config/s390x/config.common.s390x | 627 -
debian.master/config/s390x/config.flavour.generic | 3 -
debian/rules.d/1-maintainer.mk | 13 +-
debian/rules.d/2-binary-arch.mk | 6 +-
debian/rules.d/4-checks.mk | 5 +-
debian/scripts/config-check | 163 -
debian/scripts/misc/annotations | 140 +
debian/scripts/misc/kconfig/__init__.py | 0
debian/scripts/misc/kconfig/annotations.py | 242 +
debian/scripts/misc/kernelconfig | 167 +-
debian/scripts/misc/splitconfig.pl | 107 -
debian/scripts/misc/tristate.sh | 26 -
26 files changed, 4092 insertions(+), 21240 deletions(-)
delete mode 100644 debian.master/config/amd64/config.common.amd64
delete mode 100644 debian.master/config/amd64/config.flavour.generic
delete mode 100644 debian.master/config/arm64/config.common.arm64
delete mode 100644 debian.master/config/arm64/config.flavour.generic
delete mode 100644 debian.master/config/arm64/config.flavour.generic-64k
delete mode 100644 debian.master/config/armhf/config.common.armhf
delete mode 100644 debian.master/config/armhf/config.flavour.generic
delete mode 100644 debian.master/config/armhf/config.flavour.generic-lpae
delete mode 100644 debian.master/config/config.common.ubuntu
delete mode 100644 debian.master/config/ppc64el/config.common.ppc64el
delete mode 100644 debian.master/config/ppc64el/config.flavour.generic
delete mode 100644 debian.master/config/riscv64/config.common.riscv64
delete mode 100644 debian.master/config/riscv64/config.flavour.generic
delete mode 100644 debian.master/config/s390x/config.common.s390x
delete mode 100644 debian.master/config/s390x/config.flavour.generic
delete mode 100755 debian/scripts/config-check
create mode 100755 debian/scripts/misc/annotations
create mode 100644 debian/scripts/misc/kconfig/__init__.py
create mode 100644 debian/scripts/misc/kconfig/annotations.py
delete mode 100755 debian/scripts/misc/splitconfig.pl
delete mode 100755 debian/scripts/misc/tristate.sh
More information about the kernel-team
mailing list