[Bug 1863873] Re: Systemd fails to configure bridged network in LXC container
Thomas Wien
1863873 at bugs.launchpad.net
Mon Mar 2 12:06:53 UTC 2020
Thanks a lot for your explanations and workaround examples.
I expanded our ansible update playbooks to check if
/etc/netplan/10-lxc.yaml is installed, then to remove that file, runs
"netplan apply" and then remove the package "netplan.io". It seems to
work in our environment, without loosing the network configuration while
updating and also after reboot.
Here is the ansible section we tested. Maybe it is helpful to anyone
else. This script comes with no warranty and should be tested in your
environment thoroughly.
- name: Is Netplan configuring LXC?
stat:
path: /etc/netplan/10-lxc.yaml
register: netplan
- name: Remove networkd/netplan configuration
file:
path: /etc/netplan/10-lxc.yaml
state: absent
when: "netplan.stat.exists == true and ansible_virtualization_type == 'lxc'"
- name: Reload networkd/netplan
command: netplan apply
when: "netplan.stat.exists == true and ansible_virtualization_type == 'lxc'"
- name: Remove netplan.io
apt:
name: netplan.io
state: absent
when: "netplan.stat.exists == true and ansible_virtualization_type == 'lxc'"
--
You received this bug notification because you are a member of Ubuntu
Foundations Bugs, which is subscribed to systemd in Ubuntu.
https://bugs.launchpad.net/bugs/1863873
Title:
Systemd fails to configure bridged network in LXC container
Status in systemd package in Ubuntu:
Invalid
Bug description:
In all our LXC containers running Bionic Beaver, Eoan Ermine or Focal
Fossa, installing the latest systemd package results in losing network
configuration.
It is still possible to configure the network "by hand" with
/usr/sbin/ip, but of course, the configuration is lost at reboot.
An example is provided, followed by a complete procedure to reproduce
the issue.
Affected container distributions
================================
Xenial Xerus systemd 229-4ubuntu21.27: OK, not affected
Bionic Beaver systemd 237-3ubuntu10.38: OK, not affected
Bionic Beaver systemd 237-3ubuntu10.39: BUGGY
Disco Dingo systemd 240-6ubuntu5.8: OK, not affected
Eoan Ermine systemd 242-7ubuntu3.6: OK, not affected
Eoan Ermine systemd 242-7ubuntu3.7: BUGGY
Focal Fossa systemd 244.2-1ubuntu1: BUGGY
Affected hosts
==============
Debian Buster with default 4.19.0-6-amd64, custom 5.3.9, 5.4.8 or 5.4.13 kernel
Ubuntu 16.04 lxc 2.0.8-0ubuntu1~16.04.2 (https://bugs.launchpad.net/ubuntu/+source/systemd/+bug/1863873/comments/7)
Example
=======
Example host bridge configuration
---------------------------------
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq master br0 state UP group default qlen 1000
link/ether 00:25:90:2b:f1:60 brd ff:ff:ff:ff:ff:ff
3: eth1: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc mq master br1 state DOWN group default qlen 1000
link/ether 00:25:90:2b:f1:61 brd ff:ff:ff:ff:ff:ff
4: br0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
link/ether 00:25:90:2b:f1:60 brd ff:ff:ff:ff:ff:ff
inet 192.168.252.24/24 brd 192.168.252.255 scope global br0
valid_lft forever preferred_lft forever
inet 192.168.193.203/24 brd 192.168.193.255 scope global br0:1
valid_lft forever preferred_lft forever
inet6 fe80::225:90ff:fe2b:f160/64 scope link
valid_lft forever preferred_lft forever
Example container network configuration
---------------------------------------
lxc.net.0.type = veth
lxc.net.0.veth.pair = vps525389
lxc.net.0.flags = up
lxc.net.0.link = br0
lxc.net.0.hwaddr = 02:00:00:52:53:89
lxc.net.0.name = eth0
lxc.net.0.ipv4.gateway = 192.168.252.1
lxc.net.0.ipv4.address = 192.168.252.177/32
Example steps to reproduce, inside the container
------------------------------------------------
root at vps525389:~# lsb_release -rd
Description: Ubuntu 18.04.4 LTS
Release: 18.04
root at vps525389:~# apt-cache policy systemd
systemd:
Installed: 237-3ubuntu10.38
Candidate: 237-3ubuntu10.39
Version table:
237-3ubuntu10.39 500
500 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 Packages
*** 237-3ubuntu10.38 500
500 http://security.ubuntu.com/ubuntu bionic-security/main amd64 Packages
100 /var/lib/dpkg/status
237-3ubuntu10 500
500 http://archive.ubuntu.com/ubuntu bionic/main amd64 Packages
root at vps525389:~# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
1958: eth0 at if1959: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
link/ether 02:00:00:52:53:89 brd ff:ff:ff:ff:ff:ff link-netnsid 0
inet 192.168.252.177/32 brd 255.255.255.255 scope global eth0
valid_lft forever preferred_lft forever
inet6 xxxx:xxxx:x:xx::x:xxxx/128 scope global
valid_lft forever preferred_lft forever
inet6 xxxx::xx:xxxx:xxxx/64 scope link
valid_lft forever preferred_lft forever
root at vps525389:~# apt install systemd
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following additional packages will be installed:
libnss-systemd libpam-systemd libsystemd0
Suggested packages:
systemd-container policykit-1
The following packages will be upgraded:
libnss-systemd libpam-systemd libsystemd0 systemd
4 upgraded, 0 newly installed, 0 to remove and 1 not upgraded.
Need to get 3330 kB of archives.
After this operation, 7168 B of additional disk space will be used.
Do you want to continue? [Y/n]
Get:1 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libnss-systemd amd64 237-3ubuntu10.39 [104 kB]
Get:2 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libpam-systemd amd64 237-3ubuntu10.39 [107 kB]
Get:3 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 systemd amd64 237-3ubuntu10.39 [2912 kB]
Get:4 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libsystemd0 amd64 237-3ubuntu10.39 [206 kB]
Fetched 3330 kB in 3s (1274 kB/s)
(Reading database ... 18195 files and directories currently installed.)
Preparing to unpack .../libnss-systemd_237-3ubuntu10.39_amd64.deb ...
Unpacking libnss-systemd:amd64 (237-3ubuntu10.39) over (237-3ubuntu10.38) ...
Preparing to unpack .../libpam-systemd_237-3ubuntu10.39_amd64.deb ...
Unpacking libpam-systemd:amd64 (237-3ubuntu10.39) over (237-3ubuntu10.38) ...
Preparing to unpack .../systemd_237-3ubuntu10.39_amd64.deb ...
Unpacking systemd (237-3ubuntu10.39) over (237-3ubuntu10.38) ...
Preparing to unpack .../libsystemd0_237-3ubuntu10.39_amd64.deb ...
Unpacking libsystemd0:amd64 (237-3ubuntu10.39) over (237-3ubuntu10.38) ...
Setting up libsystemd0:amd64 (237-3ubuntu10.39) ...
Setting up systemd (237-3ubuntu10.39) ...
Setting up libnss-systemd:amd64 (237-3ubuntu10.39) ...
Setting up libpam-systemd:amd64 (237-3ubuntu10.39) ...
Processing triggers for dbus (1.12.2-1ubuntu1.1) ...
Processing triggers for libc-bin (2.27-3ubuntu1) ...
root at vps525389:~# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
1958: eth0 at if1959: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
link/ether 02:00:00:52:53:89 brd ff:ff:ff:ff:ff:ff link-netnsid 0
inet6 fe80::ff:fe52:5389/64 scope link
valid_lft forever preferred_lft forever
Complete procedure to reproduce the issue
=========================================
It is here assumed that there is a DHCP server available elsewhere on
the network.
Set-up
------
1. Install an amd64 Debian Buster (default network install),
2. create a bridge on the host with a static IP and deactivate DHCP, in `/etc/network/interfaces`,
```
# This file describes the network interfaces available on your system
# and how to activate them. For more information, see interfaces(5).
source /etc/network/interfaces.d/*
# The loopback network interface
auto lo
iface lo inet loopback
## The primary network interface
#allow-hotplug ens18
#iface ens18 inet dhcp
## This is an autoconfigured IPv6 interface
#iface ens18 inet6 auto
iface ens18 inet manual
auto br0
iface br0 inet static
address 192.168.1.168
netmask 255.255.255.0
gateway 192.168.1.220
bridge_ports ens18
```
3. reboot the host,
```bash
reboot
```
4. install lxc and create a bionic amd64 container,
```bash
apt install lxc
lxc-create -t download -n bionic
```
5. on the host, modify the network configuration of the container to use the bridge with a static IP in `/var/lib/lxc/bionic/config`,
```
# Template used to create this container: /usr/share/lxc/templates/lxc-download
# Parameters passed to the template:
# Template script checksum (SHA-1): 273c51343604eb85f7e294c8da0a5eb769d648f3
# For additional config options, please look at lxc.container.conf(5)
# Uncomment the following line to support nesting containers:
#lxc.include = /usr/share/lxc/config/nesting.conf
# (Be aware this has security implications)
# Distribution configuration
lxc.include = /usr/share/lxc/config/common.conf
# For Ubuntu 14.04
lxc.mount.entry = /sys/kernel/debug sys/kernel/debug none bind,optional 0 0
lxc.mount.entry = /sys/kernel/security sys/kernel/security none bind,optional 0 0
lxc.mount.entry = /sys/fs/pstore sys/fs/pstore none bind,optional 0 0
lxc.mount.entry = mqueue dev/mqueue mqueue rw,relatime,create=dir,optional 0 0
lxc.arch = linux64
# Container specific configuration
lxc.apparmor.profile = generated
lxc.apparmor.allow_nesting = 1
lxc.rootfs.path = dir:/var/lib/lxc/bionic/rootfs
lxc.uts.name = bionic
## Network configuration
#lxc.net.0.type = empty
# Network configuration
lxc.net.0.type = veth
lxc.net.0.flags = up
lxc.net.0.link = br0
lxc.net.0.name = eth0
lxc.net.0.ipv4.gateway = 192.168.1.220
lxc.net.0.ipv4.address = 192.168.1.169/32
```
6. inside the container, install the systemd packages without the bug, and deactivate DHCP in `/etc/netplan/10-lxc.yaml`,
```bash
lxc-start -n bionic
lxc-attach -n bionic
apt install systemd=237-3ubuntu10.38 libsystemd0=237-3ubuntu10.38 libnss-systemd=237-3ubuntu10.38 libpam-systemd=237-3ubuntu10.38
sed -i 's/true/false/' /etc/netplan/10-lxc.yaml
exit
```
7. stop the container.
```bash
lxc-stop -n bionic
```
Let’s do it
-----------
1. Start the container and check the IP config, which should be ok,
```bash
lxc-start -n bionic
lxc-attach -n bionic
ip a
```
2. upgrade the system and check the IP config, the static IP is gone.
```bash
apt upgrade
ip a
exit
```
If systemd is downgraded again to 237-3ubuntu10.38, the IP is back at
the next reboot of the container.
To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/systemd/+bug/1863873/+subscriptions
More information about the foundations-bugs
mailing list