[Bug 1775566] Re: networkd not applying config written by netplan on boot in q35 (PCIe) KVM guests
Christian Ehrhardt
1775566 at bugs.launchpad.net
Thu Jun 7 11:10:12 UTC 2018
Even "netplan apply" with NO yaml file fixes it.
So running in the static networkd config (with the files I copied formerly created by netplan as outlied above).
Then with debug enabled it does:
$ networkctl list
IDX LINK TYPE OPERATIONAL SETUP
1 lo loopback carrier unmanaged
2 enp0s3 ether off unmanaged
$ sudo netplan --debug apply
DEBUG:no netplan generated networkd configuration exists
DEBUG:no netplan generated NM configuration exists
DEBUG:replug enp0s3: unbinding virtio0 from /sys/bus/virtio/drivers/virtio_net
DEBUG:replug enp0s3: rebinding virtio0 to /sys/bus/virtio/drivers/virtio_net
$ networkctl list
IDX LINK TYPE OPERATIONAL SETUP
1 lo loopback carrier unmanaged
8 enp3 ether routable configured
So I did after a reboot:
echo virtio0 | sudo tee /sys/bus/virtio/drivers/virtio_net/virtio0/driver/unbind
echo virtio0 | sudo tee /sys/bus/virtio/drivers/virtio_net/bind
$ networkctl list
IDX LINK TYPE OPERATIONAL SETUP
6 enp3 ether routable configured
So it is working after the replug - adn this replug was the magic done by "netplan apply".
Now why this fails with just a static networkd config - I don't know.
But I'd say it is a networkd bug right?
The former event existed on boot
[ 0.726055] virtio_net virtio0 enp0s3: renamed from eth0
This is the one networkd should have picked up.
[ 481.972268] virtio_net virtio0 enp3: renamed from eth0
[ 481.983710] IPv6: ADDRCONF(NETDEV_UP): enp3: link is not ready
[ 482.976135] IPv6: ADDRCONF(NETDEV_CHANGE): enp3: link becomes ready
I don't know where/how to adress this further, but I hope the repro and
debug helps to fix this.
** Summary changed:
- networkd not applying config written by netplan on boot in q35 (PCIe) KVM guests
+ networkd not applying config - missing events?
** Description changed:
Hi,
- I miss some hidden trigger of "netplan apply" to understand the following case.
+
+ TL;DR:
+ - networkd config written by netplan
+ - it seems we can eliminate netplan from this and still have the issue
+ - networkd seems to miss the event of the devices and therefore consider them unmanaged
+ - rebinding them makes it work
+ - the way to trigger this I found so far are q35 KVM guests (PCIe), but
+ there might be more
+
+ ---
+
+ I miss some hidden trigger of "netplan apply" to understand the
+ following case.
I have kvm guests, you can spawn your own one to reproduce via:
- $ uvt-simplestreams-libvirt --verbose sync --source http://cloud-images.ubuntu.com/daily arch=amd64 release=bionic label=daily
- $ uvt-kvm create --password ubuntu bionic-netplan arch=amd64 release=bionic label=daily
+ $ uvt-simplestreams-libvirt --verbose sync --source http://cloud-images.ubuntu.com/daily arch=amd64 release=bionic label=daily
+ $ uvt-kvm create --password ubuntu bionic-netplan arch=amd64 release=bionic label=daily
So far all is good, but I wanted to run on q35 type guests (that means PCIe instead of PCI) based and more modern. To do so:
1. shut down your guest
2. run virsh edit bionic-netplan
2.1 replace pc-i440fx-bionic -> pc-q35-bionic
2.2 replace pci-root with pcie-root
2.3 replace piix3-uhci -> piix4-uhci
3. start the guest again
- virsh start bionic-netplan
+ virsh start bionic-netplan
It won't get network connection, this is where I started debugging.
I thought the devices might be wrong now or anything like it, but it is more puzzling.
First I realized that the device names changed from ens3 -> enp0s3 (the kernel naming).
So I thought this entry might have a problem:
- ethernets:
- ens3:
- dhcp4: true
- match:
- macaddress: 52:54:00:68:4b:62
+ ethernets:
+ ens3:
+ dhcp4: true
+ match:
+ macaddress: 52:54:00:68:4b:62
I tried to name these enp0s3 to match,but it didn't matter and also according to the netplan man page:
- If there are match: rules, then the ID field is a purely opaque name which is only being used
- for references from definitions of compound devices in the config
+ If there are match: rules, then the ID field is a purely opaque name which is only being used
+ for references from definitions of compound devices in the config
And I found it works just fine when I run "sudo netplan apply".
-
This was odd, so to summarize up to here:
- PCIe based virt guest
- netplan egenrated config not working after (re)boot
- "netplan apply" makes it working
-
I disabled any cloud init things as recommended by the comment
- /etc/cloud/cloud.cfg.d/99-disable-network-config.cfg
- network: {config: disabled}
+ /etc/cloud/cloud.cfg.d/99-disable-network-config.cfg
+ network: {config: disabled}
So that I can rely on my netplan yaml to stay as is.
I tried various things but so far can't find what magic "netplan apply" does which is missing to my boot.
-
I checked after reboot the devices are considered unmanaged by networkctl
$ networkctl list
- IDX LINK TYPE OPERATIONAL SETUP
- 1 lo loopback carrier unmanaged
- 2 enp0s3 ether off unmanaged
+ IDX LINK TYPE OPERATIONAL SETUP
+ 1 lo loopback carrier unmanaged
+ 2 enp0s3 ether off unmanaged
But the config was generated:
$ ll /run/systemd/network/
total 32
drwxr-xr-x 2 root root 200 Jun 7 09:02 ./
drwxr-xr-x 22 root root 500 Jun 7 09:02 ../
-rw-r--r-- 1 root root 69 Jun 7 09:02 10-netplan-enp0s3.link
-rw-r--r-- 1 root root 104 Jun 7 09:02 10-netplan-enp0s3.network
I checked the log and saw that apply restarts networkd.
So I thought might just restart networkd, so I ran
- $ sudo systemctl restart systemd-networkd.service
+ $ sudo systemctl restart systemd-networkd.service
But things stayed as-is without the config being picked up.
With some nice discussion and help on IRC I also tried to disable netplan and check if this is networkd only.
# make this static networkd
$ sudo cp /run/systemd/network/10-netplan-enp0s* /etc/systemd/network/
# no netplan config
$ sudo mv /etc/netplan/* /root
That was supposed to show if networkd itself (or its config files) had issues.
And with that it still did not work, so is the error in networkd instead?
If so what magic thing does "netplan apply" do to fix it?
--
You received this bug notification because you are a member of Ubuntu
Foundations Bugs, which is subscribed to systemd in Ubuntu.
https://bugs.launchpad.net/bugs/1775566
Title:
networkd not applying config - missing events?
Status in netplan.io package in Ubuntu:
New
Status in systemd package in Ubuntu:
New
Bug description:
Hi,
TL;DR:
- networkd config written by netplan
- it seems we can eliminate netplan from this and still have the issue
- networkd seems to miss the event of the devices and therefore consider them unmanaged
- rebinding them makes it work
- the way to trigger this I found so far are q35 KVM guests (PCIe), but
there might be more
---
I miss some hidden trigger of "netplan apply" to understand the
following case.
I have kvm guests, you can spawn your own one to reproduce via:
$ uvt-simplestreams-libvirt --verbose sync --source http://cloud-images.ubuntu.com/daily arch=amd64 release=bionic label=daily
$ uvt-kvm create --password ubuntu bionic-netplan arch=amd64 release=bionic label=daily
So far all is good, but I wanted to run on q35 type guests (that means PCIe instead of PCI) based and more modern. To do so:
1. shut down your guest
2. run virsh edit bionic-netplan
2.1 replace pc-i440fx-bionic -> pc-q35-bionic
2.2 replace pci-root with pcie-root
2.3 replace piix3-uhci -> piix4-uhci
3. start the guest again
virsh start bionic-netplan
It won't get network connection, this is where I started debugging.
I thought the devices might be wrong now or anything like it, but it is more puzzling.
First I realized that the device names changed from ens3 -> enp0s3 (the kernel naming).
So I thought this entry might have a problem:
ethernets:
ens3:
dhcp4: true
match:
macaddress: 52:54:00:68:4b:62
I tried to name these enp0s3 to match,but it didn't matter and also according to the netplan man page:
If there are match: rules, then the ID field is a purely opaque name which is only being used
for references from definitions of compound devices in the config
And I found it works just fine when I run "sudo netplan apply".
This was odd, so to summarize up to here:
- PCIe based virt guest
- netplan egenrated config not working after (re)boot
- "netplan apply" makes it working
I disabled any cloud init things as recommended by the comment
/etc/cloud/cloud.cfg.d/99-disable-network-config.cfg
network: {config: disabled}
So that I can rely on my netplan yaml to stay as is.
I tried various things but so far can't find what magic "netplan apply" does which is missing to my boot.
I checked after reboot the devices are considered unmanaged by networkctl
$ networkctl list
IDX LINK TYPE OPERATIONAL SETUP
1 lo loopback carrier unmanaged
2 enp0s3 ether off unmanaged
But the config was generated:
$ ll /run/systemd/network/
total 32
drwxr-xr-x 2 root root 200 Jun 7 09:02 ./
drwxr-xr-x 22 root root 500 Jun 7 09:02 ../
-rw-r--r-- 1 root root 69 Jun 7 09:02 10-netplan-enp0s3.link
-rw-r--r-- 1 root root 104 Jun 7 09:02 10-netplan-enp0s3.network
I checked the log and saw that apply restarts networkd.
So I thought might just restart networkd, so I ran
$ sudo systemctl restart systemd-networkd.service
But things stayed as-is without the config being picked up.
With some nice discussion and help on IRC I also tried to disable netplan and check if this is networkd only.
# make this static networkd
$ sudo cp /run/systemd/network/10-netplan-enp0s* /etc/systemd/network/
# no netplan config
$ sudo mv /etc/netplan/* /root
That was supposed to show if networkd itself (or its config files) had issues.
And with that it still did not work, so is the error in networkd instead?
If so what magic thing does "netplan apply" do to fix it?
To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/netplan.io/+bug/1775566/+subscriptions
More information about the foundations-bugs
mailing list