[Bug 2046486] Re: units with credentials fail in LXD containers
Launchpad Bug Tracker
2046486 at bugs.launchpad.net
Fri Jul 12 06:35:20 UTC 2024
Status changed to 'Confirmed' because the bug affects multiple users.
** Changed in: lxd (Ubuntu)
Status: New => Confirmed
--
You received this bug notification because you are a member of Ubuntu
Foundations Bugs, which is subscribed to systemd in Ubuntu.
https://bugs.launchpad.net/bugs/2046486
Title:
units with credentials fail in LXD containers
Status in lxd:
New
Status in lxd package in Ubuntu:
Confirmed
Status in samba package in Ubuntu:
Fix Released
Status in systemd package in Ubuntu:
Triaged
Bug description:
Many units shipped by systemd use credentials in some way by default
now (in v256). So this issue is now about much more than the original
test case failure.
For example,
root at oracular:~# apt policy systemd
systemd:
Installed: 256-1ubuntu1
Candidate: 256-1ubuntu1
Version table:
*** 256-1ubuntu1 100
100 http://archive.ubuntu.com/ubuntu oracular-proposed/main amd64 Packages
100 /var/lib/dpkg/status
255.4-1ubuntu8 500
500 http://archive.ubuntu.com/ubuntu oracular/main amd64 Packages
root at oracular:~# for service in $(find /usr/lib/systemd/system -maxdepth 1 -name "systemd-*.service"); do grep -q "Credential.*=" "$service" && echo "$service"; done
/usr/lib/systemd/system/systemd-sysusers.service
/usr/lib/systemd/system/systemd-resolved.service
/usr/lib/systemd/system/systemd-firstboot.service
/usr/lib/systemd/system/systemd-network-generator.service
/usr/lib/systemd/system/systemd-journald.service
/usr/lib/systemd/system/systemd-sysctl.service
/usr/lib/systemd/system/systemd-tmpfiles-setup-dev-early.service
/usr/lib/systemd/system/systemd-tmpfiles-setup-dev.service
/usr/lib/systemd/system/systemd-tmpfiles-setup.service
/usr/lib/systemd/system/systemd-udev-load-credentials.service
/usr/lib/systemd/system/systemd-tmpfiles-clean.service
/usr/lib/systemd/system/systemd-networkd.service
root at oracular:~# systemctl status systemd-sysusers.service systemd-resolved.service systemd-firstboot.service systemd-network-generator.service systemd-journald.service systemd-sysctl.service systemd-tmpfiles-setup-dev-early.service systemd-tmpfiles-setup-dev.service systemd-tmpfiles-setup.service systemd-udev-load-credentials.service systemd-tmpfiles-clean.service systemd-networkd.service
○ systemd-sysusers.service - Create System Users
Loaded: loaded (/usr/lib/systemd/system/systemd-sysusers.service; static)
Active: inactive (dead)
Condition: start condition unmet at Mon 2024-06-24 18:58:48 UTC; 1min 0s ago
├─ ConditionNeedsUpdate=|/etc was not met
└─ ConditionCredential=|sysusers.extra was not met
Docs: man:sysusers.d(5)
man:systemd-sysusers.service(8)
× systemd-resolved.service - Network Name Resolution
Loaded: loaded (/usr/lib/systemd/system/systemd-resolved.service; enabled; preset: enabled)
Active: failed (Result: exit-code) since Mon 2024-06-24 18:58:49 UTC; 59s ago
Invocation: b1aaa662750f48868fe3388e4524c462
Docs: man:systemd-resolved.service(8)
man:org.freedesktop.resolve1(5)
https://systemd.io/WRITING_NETWORK_CONFIGURATION_MANAGERS
https://systemd.io/WRITING_RESOLVER_CLIENTS
Process: 258 ExecStart=/usr/lib/systemd/systemd-resolved (code=exited, status=243/CREDENTIALS)
Main PID: 258 (code=exited, status=243/CREDENTIALS)
○ systemd-firstboot.service - First Boot Wizard
Loaded: loaded (/usr/lib/systemd/system/systemd-firstboot.service; static)
Active: inactive (dead)
Condition: start condition unmet at Mon 2024-06-24 18:58:48 UTC; 59s ago
└─ ConditionFirstBoot=yes was not met
Docs: man:systemd-firstboot(1)
○ systemd-network-generator.service - Generate network units from Kernel command line
Loaded: loaded (/usr/lib/systemd/system/systemd-network-generator.service; disabled; preset: enabled)
Active: inactive (dead)
Docs: man:systemd-network-generator.service(8)
× systemd-journald.service - Journal Service
Loaded: loaded (/usr/lib/systemd/system/systemd-journald.service; static)
Drop-In: /usr/lib/systemd/system/systemd-journald.service.d
└─nice.conf
Active: failed (Result: exit-code) since Mon 2024-06-24 18:58:48 UTC; 1min 0s ago
Invocation: 7caace7a15c749f3a86fb15fcfb94dff
TriggeredBy: × systemd-journald-dev-log.socket
× systemd-journald.socket
○ systemd-journald-audit.socket
Docs: man:systemd-journald.service(8)
man:journald.conf(5)
Process: 124 ExecStart=/usr/lib/systemd/systemd-journald (code=exited, status=243/CREDENTIALS)
Main PID: 124 (code=exited, status=243/CREDENTIALS)
FD Store: 0 (limit: 4224)
× systemd-sysctl.service - Apply Kernel Variables
Loaded: loaded (/usr/lib/systemd/system/systemd-sysctl.service; static)
Active: failed (Result: exit-code) since Mon 2024-06-24 18:58:48 UTC; 1min 0s ago
Invocation: 5e90310a27b043ceae80c96e35c41451
Docs: man:systemd-sysctl.service(8)
man:sysctl.d(5)
Process: 97 ExecStart=/usr/lib/systemd/systemd-sysctl (code=exited, status=243/CREDENTIALS)
Main PID: 97 (code=exited, status=243/CREDENTIALS)
× systemd-tmpfiles-setup-dev-early.service - Create Static Device Nodes in /dev gracefully
Loaded: loaded (/usr/lib/systemd/system/systemd-tmpfiles-setup-dev-early.service; static)
Active: failed (Result: exit-code) since Mon 2024-06-24 18:58:48 UTC; 1min 0s ago
Invocation: 78e3c68cfa9a4a7982950b08c0f1385f
Docs: man:tmpfiles.d(5)
man:systemd-tmpfiles(8)
Process: 73 ExecStart=systemd-tmpfiles --prefix=/dev --create --boot --graceful (code=exited, status=243/CREDENTIALS)
Main PID: 73 (code=exited, status=243/CREDENTIALS)
× systemd-tmpfiles-setup-dev.service - Create Static Device Nodes in /dev
Loaded: loaded (/usr/lib/systemd/system/systemd-tmpfiles-setup-dev.service; static)
Active: failed (Result: exit-code) since Mon 2024-06-24 18:58:48 UTC; 1min 0s ago
Invocation: 46458c7b6e134ef8be299900db7cc288
Docs: man:tmpfiles.d(5)
man:systemd-tmpfiles(8)
Process: 98 ExecStart=systemd-tmpfiles --prefix=/dev --create --boot (code=exited, status=243/CREDENTIALS)
Main PID: 98 (code=exited, status=243/CREDENTIALS)
× systemd-tmpfiles-setup.service - Create Volatile Files and Directories
Loaded: loaded (/usr/lib/systemd/system/systemd-tmpfiles-setup.service; static)
Active: failed (Result: exit-code) since Mon 2024-06-24 18:58:48 UTC; 1min 0s ago
Invocation: f4e64afdc8774170a9b29b8cf2919f46
Docs: man:tmpfiles.d(5)
man:systemd-tmpfiles(8)
Process: 147 ExecStart=systemd-tmpfiles --create --remove --boot --exclude-prefix=/dev (code=exited, status=243/CREDENTIALS)
Main PID: 147 (code=exited, status=243/CREDENTIALS)
× systemd-udev-load-credentials.service - Load udev Rules from Credentials
Loaded: loaded (/usr/lib/systemd/system/systemd-udev-load-credentials.service; disabled; preset: enabled)
Active: failed (Result: exit-code) since Mon 2024-06-24 18:58:48 UTC; 1min 0s ago
Invocation: cb5a1f43cde248de80fcf701b4b5d381
Docs: man:udevadm(8)
man:udev(7)
man:systemd.system-credentials(7)
Process: 75 ExecStart=udevadm control --load-credentials (code=exited, status=243/CREDENTIALS)
Main PID: 75 (code=exited, status=243/CREDENTIALS)
○ systemd-tmpfiles-clean.service - Cleanup of Temporary Directories
Loaded: loaded (/usr/lib/systemd/system/systemd-tmpfiles-clean.service; static)
Active: inactive (dead)
TriggeredBy: ● systemd-tmpfiles-clean.timer
Docs: man:tmpfiles.d(5)
man:systemd-tmpfiles(8)
× systemd-networkd.service - Network Configuration
Loaded: loaded (/usr/lib/systemd/system/systemd-networkd.service; enabled; preset: enabled)
Active: failed (Result: exit-code) since Mon 2024-06-24 18:58:49 UTC; 59s ago
Invocation: 5d960369ea944d5cbac4382e42ded1d0
TriggeredBy: × systemd-networkd.socket
Docs: man:systemd-networkd.service(8)
man:org.freedesktop.network1(5)
Process: 280 ExecStart=/usr/lib/systemd/systemd-networkd (code=exited, status=243/CREDENTIALS)
Main PID: 280 (code=exited, status=243/CREDENTIALS)
FD Store: 0 (limit: 512)
[Original Description]
To demonstrate this, in an unprivileged LXD container, create the
following unit (taken from the systemd test suite):
$ cat > /etc/systemd/system/exec-set-credential.service << EOF
# SPDX-License-Identifier: LGPL-2.1-or-later
[Unit]
Description=Test for SetCredential=
[Service]
ExecStart=/bin/sh -x -c 'test "$$(cat %d/test-execute.set-credential)" = "hoge"'
ExecStartPost=/bin/sh -x -c 'test "$$(cat %d/test-execute.set-credential)" = "hoge"'
ExecStop=/bin/sh -x -c 'test "$$(cat %d/test-execute.set-credential)" = "hoge"'
ExecStopPost=/bin/sh -x -c 'test "$$(cat %d/test-execute.set-credential)" = "hoge"'
Type=oneshot
SetCredential=test-execute.set-credential:hoge
EOF
$ systemctl daemon-reload
$ systemctl start exec-set-credential.service
Job for exec-set-credential.service failed because the control process exited with error code.
See "systemctl status exec-set-credential.service" and "journalctl -xeu exec-set-credential.service" for details.
With debug logs enabled, we see:
$ journalctl -u exec-set-credential.service -b --no-pager
Dec 14 19:24:24 noble systemd[1]: exec-set-credential.service: Trying to enqueue job exec-set-credential.service/start/replace
Dec 14 19:24:24 noble systemd[1]: exec-set-credential.service: Installed new job exec-set-credential.service/start as 2740
Dec 14 19:24:24 noble systemd[1]: exec-set-credential.service: Enqueued job exec-set-credential.service/start as 2740
Dec 14 19:24:24 noble systemd[1]: exec-set-credential.service: Will spawn child (service_enter_start): /bin/sh
Dec 14 19:24:24 noble systemd[1]: exec-set-credential.service: Failed to set 'trusted.invocation_id' xattr on control group /system.slice/exec-set-credential.service, ignoring: Operation not permitted
Dec 14 19:24:24 noble systemd[1]: exec-set-credential.service: Failed to remove 'trusted.delegate' xattr flag on control group /system.slice/exec-set-credential.service, ignoring: Operation not permitted
Dec 14 19:24:24 noble systemd[1]: exec-set-credential.service: Failed to remove 'trusted.survive_final_kill_signal' xattr flag on control group /system.slice/exec-set-credential.service, ignoring: Operation not permitted
Dec 14 19:24:24 noble systemd[1]: exec-set-credential.service: Passing 0 fds to service
Dec 14 19:24:24 noble systemd[1]: exec-set-credential.service: About to execute: /bin/sh -x -c "test \"1031(cat /run/credentials/exec-set-credential.service/test-execute.set-credential)\" = \"hoge\""
Dec 14 19:24:24 noble systemd[1]: exec-set-credential.service: Forked /bin/sh as 2183
Dec 14 19:24:24 noble (sh)[2183]: PR_SET_MM_ARG_START failed: Operation not permitted
Dec 14 19:24:24 noble (sh)[2183]: Found cgroup2 on /sys/fs/cgroup/, full unified hierarchy
Dec 14 19:24:24 noble (sh)[2183]: Found cgroup2 on /sys/fs/cgroup/, full unified hierarchy
Dec 14 19:24:24 noble systemd[1]: exec-set-credential.service: Changed dead -> start
Dec 14 19:24:24 noble systemd[1]: Starting exec-set-credential.service - Test for SetCredential=...
Dec 14 19:24:24 noble (sh)[2183]: Successfully forked off '(sd-mkdcreds)' as PID 2184.
Dec 14 19:24:24 noble (sd-[2184]: Changing mount propagation /dev (MS_REC|MS_SLAVE "")
Dec 14 19:24:24 noble (sd-[2184]: Mounting ramfs (ramfs) on /dev/shm (MS_NOSUID|MS_NODEV|MS_NOEXEC|MS_NOSYMFOLLOW "mode=0700")...
Dec 14 19:24:24 noble (sd-[2184]: Changing mount flags /dev/shm (MS_RDONLY|MS_NOSUID|MS_NODEV|MS_NOEXEC|MS_REMOUNT|MS_NOSYMFOLLOW|MS_BIND "")...
Dec 14 19:24:24 noble (sd-[2184]: Failed to mount n/a (type n/a) on /dev/shm (MS_RDONLY|MS_NOSUID|MS_NODEV|MS_NOEXEC|MS_REMOUNT|MS_NOSYMFOLLOW|MS_BIND ""): Permission denied
Dec 14 19:24:24 noble (sh)[2183]: (sd-mkdcreds) failed with exit status 1.
Dec 14 19:24:24 noble (sh)[2183]: exec-set-credential.service: Failed to set up credentials: Protocol error
Dec 14 19:24:24 noble systemd[1]: exec-set-credential.service: Child 2183 belongs to exec-set-credential.service.
Dec 14 19:24:24 noble systemd[1]: exec-set-credential.service: Main process exited, code=exited, status=243/CREDENTIALS
Dec 14 19:24:24 noble systemd[1]: exec-set-credential.service: Will spawn child (service_enter_stop_post): /bin/sh
Dec 14 19:24:24 noble systemd[1]: exec-set-credential.service: About to execute: /bin/sh -x -c "test \"1031(cat /run/credentials/exec-set-credential.service/test-execute.set-credential)\" = \"hoge\""
Dec 14 19:24:24 noble systemd[1]: exec-set-credential.service: Forked /bin/sh as 2186
Dec 14 19:24:24 noble systemd[1]: exec-set-credential.service: Changed start -> stop-post
Dec 14 19:24:24 noble (sh)[2186]: PR_SET_MM_ARG_START failed: Operation not permitted
Dec 14 19:24:24 noble (sh)[2186]: Found cgroup2 on /sys/fs/cgroup/, full unified hierarchy
Dec 14 19:24:24 noble (sh)[2186]: Found cgroup2 on /sys/fs/cgroup/, full unified hierarchy
Dec 14 19:24:24 noble sh[2186]: + test 1031(cat /run/credentials/exec-set-credential.service/test-execute.set-credential) = hoge
Dec 14 19:24:24 noble systemd[1]: exec-set-credential.service: Child 2186 belongs to exec-set-credential.service.
Dec 14 19:24:24 noble systemd[1]: exec-set-credential.service: Control process exited, code=exited, status=1/FAILURE
Dec 14 19:24:24 noble systemd[1]: exec-set-credential.service: Got final SIGCHLD for state stop-post.
Dec 14 19:24:24 noble systemd[1]: exec-set-credential.service: Failed with result 'exit-code'.
Dec 14 19:24:24 noble systemd[1]: exec-set-credential.service: Service will not restart (restart setting)
Dec 14 19:24:24 noble systemd[1]: exec-set-credential.service: Changed stop-post -> failed
Dec 14 19:24:24 noble systemd[1]: exec-set-credential.service: Job 2740 exec-set-credential.service/start finished, result=failed
Dec 14 19:24:24 noble systemd[1]: Failed to start exec-set-credential.service - Test for SetCredential=.
Dec 14 19:24:24 noble systemd[1]: exec-set-credential.service: Unit entered failed state.
Dec 14 19:24:24 noble systemd[1]: exec-set-credential.service: Consumed 23ms CPU time.
Dec 14 19:24:24 noble systemd[1]: exec-set-credential.service: Releasing resources...
To manage notifications about this bug go to:
https://bugs.launchpad.net/lxd/+bug/2046486/+subscriptions
More information about the foundations-bugs
mailing list