[Bug 2016908] Re: Unable to deploy hosts with lunar images after 20230319 - fails to connect and download squashfs
Dimitri John Ledkov
2016908 at bugs.launchpad.net
Thu Apr 20 19:41:35 UTC 2023
** Description changed:
I'm assuming the image being used for these deploys is 20230417 or
20230417.1 based on the fact that I saw a 6.2 kernel being used which I
don't believe was part of the 20230319 serial. I don't have access to
the maas server, so I can't directly check any log files.
MAAS Version: 3.3.2
Here's where the serial log indicates it can't download the squashfs. The full log is attached as scobee-lunar-no-squashfs.log (there are some other console message intermixed):
no search or nameservers found in /run/net-BOOTIF.conf /run/net-*.conf /run/net6
-*.conf
:: root=squash:http://10.229.32.21:5248/images/ubuntu/arm64/ga-23.04/lunar/candi
date/squa[ 206.804704] Btrfs loaded, crc32c=crc32c-generic, zoned=yes, fsverity
=yes
shfs
:: mount_squash downloading http://10.229.32.21:5248/images/ubuntu/arm64/ga-23.0
4/lunar/candidate/squashfs to /root.tmp.img
Connecting to 10.229.32.21:5248 (10.229.32.21:5248)
wget: can't connect to remote host (10.229.32.21): Network is unreachable
:: mount -t squashfs -o loop '/root.tmp.img' '/root.tmp'
mount: mounting /root.tmp.img on /root.tmp failed: No such file or directory
done.
+ Still gathering logs and info and will update as I go.
- Still gathering logs and info and will update as I go.
+
+ ----
+ Kernel Bug / Apparmor
+ reproducer
+
+ $ wget https://images.maas.io/ephemeral-v3/candidate/lunar/amd64/20230419/ga-23.04/generic/boot-kernel
+ $ wget https://images.maas.io/ephemeral-v3/candidate/lunar/amd64/20230419/ga-23.04/generic/boot-initrd
+ $ qemu-system-x86_64 -nographic -m 2G -kernel ./boot-kernel -initrd ./boot-initrd -append 'console=ttyS0 break=modules apparmor=0'
+
+
+ #start the VM
+ ....
+ Starting systemd-udevd version 252.5-2ubuntu3
+ Spawning shell within the initramfs
+
+
+ BusyBox v1.35.0 (Ubuntu 1:1.35.0-4ubuntu1) built-in shell (ash)
+ Enter 'help' for a list of built-in commands.
+
+ (initramfs) udevadm info --export-db
+ Failed to set death signal: Invalid argument
+
+ Observe that udevadm fails to setup death signal, with in systemd code
+ is this
+
+ https://github.com/systemd/systemd/blob/08c2f9c626e0f0052d505b1b7e52f335c0fbfa1d/src/basic/process-
+ util.c#L1252
+
+ if (flags & (FORK_DEATHSIG|FORK_DEATHSIG_SIGINT))
+ if (prctl(PR_SET_PDEATHSIG, (flags & FORK_DEATHSIG_SIGINT) ? SIGINT : SIGTERM) < 0) {
+ log_full_errno(prio, errno, "Failed to set death signal: %m");
+ _exit(EXIT_FAILURE);
+ }
+
+ ----
+
+ MAAS bug
+ Why is maas setting `apparmor=0` ? Ubuntu shouldn't be used without apparmor. Even for deployment and commisioning.
** Changed in: linux (Ubuntu)
Status: Incomplete => Triaged
** Changed in: maas-images
Status: Incomplete => Invalid
** Changed in: systemd (Ubuntu)
Status: New => Invalid
** Also affects: maas
Importance: Undecided
Status: New
** Summary changed:
- Unable to deploy hosts with lunar images after 20230319 - fails to connect and download squashfs
+ udev fails to make prctl() syscall with apparmor=0 (as used by maas by default)
** Description changed:
I'm assuming the image being used for these deploys is 20230417 or
20230417.1 based on the fact that I saw a 6.2 kernel being used which I
don't believe was part of the 20230319 serial. I don't have access to
the maas server, so I can't directly check any log files.
MAAS Version: 3.3.2
Here's where the serial log indicates it can't download the squashfs. The full log is attached as scobee-lunar-no-squashfs.log (there are some other console message intermixed):
no search or nameservers found in /run/net-BOOTIF.conf /run/net-*.conf /run/net6
-*.conf
:: root=squash:http://10.229.32.21:5248/images/ubuntu/arm64/ga-23.04/lunar/candi
date/squa[ 206.804704] Btrfs loaded, crc32c=crc32c-generic, zoned=yes, fsverity
=yes
shfs
:: mount_squash downloading http://10.229.32.21:5248/images/ubuntu/arm64/ga-23.0
4/lunar/candidate/squashfs to /root.tmp.img
Connecting to 10.229.32.21:5248 (10.229.32.21:5248)
wget: can't connect to remote host (10.229.32.21): Network is unreachable
:: mount -t squashfs -o loop '/root.tmp.img' '/root.tmp'
mount: mounting /root.tmp.img on /root.tmp failed: No such file or directory
done.
Still gathering logs and info and will update as I go.
-
----
Kernel Bug / Apparmor
reproducer
$ wget https://images.maas.io/ephemeral-v3/candidate/lunar/amd64/20230419/ga-23.04/generic/boot-kernel
$ wget https://images.maas.io/ephemeral-v3/candidate/lunar/amd64/20230419/ga-23.04/generic/boot-initrd
$ qemu-system-x86_64 -nographic -m 2G -kernel ./boot-kernel -initrd ./boot-initrd -append 'console=ttyS0 break=modules apparmor=0'
-
#start the VM
....
Starting systemd-udevd version 252.5-2ubuntu3
Spawning shell within the initramfs
-
BusyBox v1.35.0 (Ubuntu 1:1.35.0-4ubuntu1) built-in shell (ash)
Enter 'help' for a list of built-in commands.
(initramfs) udevadm info --export-db
Failed to set death signal: Invalid argument
Observe that udevadm fails to setup death signal, with in systemd code
is this
https://github.com/systemd/systemd/blob/08c2f9c626e0f0052d505b1b7e52f335c0fbfa1d/src/basic/process-
util.c#L1252
- if (flags & (FORK_DEATHSIG|FORK_DEATHSIG_SIGINT))
- if (prctl(PR_SET_PDEATHSIG, (flags & FORK_DEATHSIG_SIGINT) ? SIGINT : SIGTERM) < 0) {
- log_full_errno(prio, errno, "Failed to set death signal: %m");
- _exit(EXIT_FAILURE);
- }
+ if (flags & (FORK_DEATHSIG|FORK_DEATHSIG_SIGINT))
+ if (prctl(PR_SET_PDEATHSIG, (flags & FORK_DEATHSIG_SIGINT) ? SIGINT : SIGTERM) < 0) {
+ log_full_errno(prio, errno, "Failed to set death signal: %m");
+ _exit(EXIT_FAILURE);
+ }
+
+ workaround set kernel commandline to `apparmor=1`
----
MAAS bug
Why is maas setting `apparmor=0` ? Ubuntu shouldn't be used without apparmor. Even for deployment and commisioning.
--
You received this bug notification because you are a member of Ubuntu
Foundations Bugs, which is subscribed to systemd in Ubuntu.
https://bugs.launchpad.net/bugs/2016908
Title:
udev fails to make prctl() syscall with apparmor=0 (as used by maas by
default)
Status in MAAS:
New
Status in maas-images:
Invalid
Status in linux package in Ubuntu:
Triaged
Status in systemd package in Ubuntu:
Invalid
Bug description:
I'm assuming the image being used for these deploys is 20230417 or
20230417.1 based on the fact that I saw a 6.2 kernel being used which
I don't believe was part of the 20230319 serial. I don't have access
to the maas server, so I can't directly check any log files.
MAAS Version: 3.3.2
Here's where the serial log indicates it can't download the squashfs. The full log is attached as scobee-lunar-no-squashfs.log (there are some other console message intermixed):
no search or nameservers found in /run/net-BOOTIF.conf /run/net-*.conf /run/net6
-*.conf
:: root=squash:http://10.229.32.21:5248/images/ubuntu/arm64/ga-23.04/lunar/candi
date/squa[ 206.804704] Btrfs loaded, crc32c=crc32c-generic, zoned=yes, fsverity
=yes
shfs
:: mount_squash downloading http://10.229.32.21:5248/images/ubuntu/arm64/ga-23.0
4/lunar/candidate/squashfs to /root.tmp.img
Connecting to 10.229.32.21:5248 (10.229.32.21:5248)
wget: can't connect to remote host (10.229.32.21): Network is unreachable
:: mount -t squashfs -o loop '/root.tmp.img' '/root.tmp'
mount: mounting /root.tmp.img on /root.tmp failed: No such file or directory
done.
Still gathering logs and info and will update as I go.
----
Kernel Bug / Apparmor
reproducer
$ wget https://images.maas.io/ephemeral-v3/candidate/lunar/amd64/20230419/ga-23.04/generic/boot-kernel
$ wget https://images.maas.io/ephemeral-v3/candidate/lunar/amd64/20230419/ga-23.04/generic/boot-initrd
$ qemu-system-x86_64 -nographic -m 2G -kernel ./boot-kernel -initrd ./boot-initrd -append 'console=ttyS0 break=modules apparmor=0'
#start the VM
....
Starting systemd-udevd version 252.5-2ubuntu3
Spawning shell within the initramfs
BusyBox v1.35.0 (Ubuntu 1:1.35.0-4ubuntu1) built-in shell (ash)
Enter 'help' for a list of built-in commands.
(initramfs) udevadm info --export-db
Failed to set death signal: Invalid argument
Observe that udevadm fails to setup death signal, with in systemd code
is this
https://github.com/systemd/systemd/blob/08c2f9c626e0f0052d505b1b7e52f335c0fbfa1d/src/basic/process-
util.c#L1252
if (flags & (FORK_DEATHSIG|FORK_DEATHSIG_SIGINT))
if (prctl(PR_SET_PDEATHSIG, (flags & FORK_DEATHSIG_SIGINT) ? SIGINT : SIGTERM) < 0) {
log_full_errno(prio, errno, "Failed to set death signal: %m");
_exit(EXIT_FAILURE);
}
workaround set kernel commandline to `apparmor=1`
----
MAAS bug
Why is maas setting `apparmor=0` ? Ubuntu shouldn't be used without apparmor. Even for deployment and commisioning.
To manage notifications about this bug go to:
https://bugs.launchpad.net/maas/+bug/2016908/+subscriptions
More information about the foundations-bugs
mailing list