[Bug 1928259] Re: Package upgrade won't restart services
Bug Watch Updater
1928259 at bugs.launchpad.net
Wed Nov 30 20:18:15 UTC 2022
Launchpad has imported 3 comments from the remote bug at
https://bugzilla.redhat.com/show_bug.cgi?id=1961322.
If you reply to an imported comment from within Launchpad, your comment
will be sent to the remote bug automatically. Read more about
Launchpad's inter-bugtracker facilities at
https://help.launchpad.net/InterBugTracking.
------------------------------------------------------------------------
On 2021-05-17T17:16:11+00:00 andreas wrote:
Description of problem:
On an NFSv4 client, using sec=krb5 for the NFSv4 mount, the rpc.gssd daemon is needed. As far as I can tell, it's started as part of the nfs-client.target target, but it's a bit fuzzy (nfs-utils.service also deals with it). In any case, it's not meant to be enabled on its own, as it has no [Install] section.
If you then upgrade the nfs-utils rpm, it's expected that it should restart the services that are already running, but that doesn't seem to happen. I tested with fedora 33, 34, and now rawhide. They all have a postinst that tries to restart nfs-client.target. In rawhide it's a bit different in that it sets a marker, but other than that, f33 and f34 just used try-restart. Here is rawhide's nfs-utils postuninstall for the upgrade case:
postuninstall scriptlet (using /bin/sh):
if [ $1 -ge 1 ] && [ -x /usr/bin/systemctl ]; then
# Package upgrade, not uninstall
for unit in nfs-client.target; do
/usr/bin/systemctl set-property $unit Markers=+needs-restart || :
done
fi
That currently fails with (below is debug output from dnf reinstall nfs-utils):
D: %postun(nfs-utils-1:2.5.3-3.rc2.fc35.x86_64): execv(/bin/sh) pid 808
+ '[' 1 -ge 1 ']'
+ '[' -x /usr/bin/systemctl ']'
+ for unit in nfs-client.target
+ /usr/bin/systemctl set-property nfs-client.target Markers=+needs-restart
Not supported unit type
+ :
+ '[' 1 -ge 1 ']'
+ '[' -x /usr/bin/systemctl ']'
+ for unit in nfs-server
+ /usr/bin/systemctl set-property nfs-server Markers=+needs-restart
But even if I try a restart (instead of that property setting, new to rawhide), it won't touch rpc.gssd (bash prompt says f33 because that's the fedora version I started with testing, and then upgraded to 34 and now it's running rawhide):
[root at f33-client ~]# pidof rpc.gssd
447
[root at f33-client ~]# systemctl restart nfs-client.target
[root at f33-client ~]# pidof rpc.gssd
447
[root at f33-client ~]#
What does restart it is, obviously, systemctl restart rpc-gssd.service, and restart nfs-utils.service:
[root at f33-client ~]# systemctl restart rpc-gssd.service
[root at f33-client ~]# pidof rpc.gssd
937
[root at f33-client ~]# systemctl restart nfs-utils.service
[root at f33-client ~]# pidof rpc.gssd
945
[root at f33-client ~]#
nfs-utils.service is another service without an [Install] section, so it
can't be enabled. But maybe that's what should be restarted instead of
the target? Or did I miss enabling some service that I should have?
Version-Release number of selected component (if applicable):
nfs-utils-2.5.3-3.rc2.fc35.x86_64
How reproducible:
Always on upgrade.
Steps to Reproduce:
1. Configure NFSv4 kerberized mountpoint. Check that rpc.gssd is running, and that it also starts on its own on reboot
2. Take note of the rpc.gssd pid
3. Upgrade the nfs-utils package. Note that rpc.gssd was not restarted
Actual results:
rpc.gssd is not restarted after the nfs-utils package is upgraded.
Expected results:
rpc.gssd, if running, should be restarted after the package that ships it (nfs-utils) is upgraded.
Additional info:
Reply at: https://bugs.launchpad.net/ubuntu/+source/nfs-
utils/+bug/1928259/comments/1
------------------------------------------------------------------------
On 2021-08-10T13:02:47+00:00 bcotton wrote:
This bug appears to have been reported against 'rawhide' during the Fedora 35 development cycle.
Changing version to 35.
Reply at: https://bugs.launchpad.net/ubuntu/+source/nfs-
utils/+bug/1928259/comments/18
------------------------------------------------------------------------
On 2022-11-29T16:57:27+00:00 bcotton wrote:
This message is a reminder that Fedora Linux 35 is nearing its end of life.
Fedora will stop maintaining and issuing updates for Fedora Linux 35 on 2022-12-13.
It is Fedora's policy to close all bug reports from releases that are no longer
maintained. At that time this bug will be closed as EOL if it remains open with a
'version' of '35'.
Package Maintainer: If you wish for this bug to remain open because you
plan to fix it in a currently maintained version, change the 'version'
to a later Fedora Linux version.
Thank you for reporting this issue and we are sorry that we were not
able to fix it before Fedora Linux 35 is end of life. If you would still like
to see this bug fixed and are able to reproduce it against a later version
of Fedora Linux, you are encouraged to change the 'version' to a later version
prior to this bug being closed.
Reply at: https://bugs.launchpad.net/ubuntu/+source/nfs-
utils/+bug/1928259/comments/19
--
You received this bug notification because you are a member of Ubuntu
Foundations Bugs, which is subscribed to nfs-utils in Ubuntu.
https://bugs.launchpad.net/bugs/1928259
Title:
Package upgrade won't restart services
Status in nfs-utils package in Ubuntu:
Fix Released
Status in nfs-utils source package in Bionic:
Fix Released
Status in nfs-utils source package in Focal:
Fix Released
Status in nfs-utils source package in Groovy:
Fix Released
Status in nfs-utils source package in Hirsute:
Fix Released
Status in nfs-utils package in Debian:
Fix Released
Status in nfs-utils package in Fedora:
Confirmed
Bug description:
[Impact]
In order to get the fixes provided by a package update, the affected services shipped in it need to be restarted. When that restart does not happen, the system remains running the old binaries with the bug(s).
This bug was found while testing the fix for #1927745, which affected
rpc.gssd, one of the services shipped in nfs-common. Without the
restart, systems that installed the update are still affected by the
bug.
[Test Plan]
To make the test simple, we are not going to mount an NFSv4 share using kerberos. We are just going to have a minimal configuration that gets rpc.gssd running to demonstrate the before and after of this bug.
For a more thorough testing, which includes actually mounting an NFSv4
export with kerberos, follow the test instructions of bug #1927745,
and you will see that the manual restart included because of this bug
here, after the package is updated, is no longer needed.
TEST (A)
# create a VM for the affected ubuntu release under test, login and run:
sudo touch /etc/krb5.keytab
sudo chmod 0600 /etc/krb5.keytab
# install nfs-common
sudo apt install nfs-common -y
# note message about nfs-utils.service being disabled/static:
nfs-utils.service is a disabled or a static unit, not starting it.
# Manually start rpc-gssd. It will start, but since we have an empty
# krb5.keytab file, it won't work. That's ok, we are not actually going to
# mount nfsv4
sudo systemctl start rpc-gssd.service
# Check it's running, and make note of its pid:
pidof rpc.gssd
2994
# reinstall nfs-common
sudo apt install --reinstall nfs-common
# note rpc-gssd wasn't restarted
pidof rpc.gssd
2994
# install the fixed nfs-common package. Notice the message about starting a disabled or static unit no longer appears:
sudo apt install nfs-common
# this time, rpc.gssd is restarted
pidof rpc.gssd
5000
TEST (B)
This test is to confirm no new services are started after the fixed package is installed for the first time.
# create a VM for the affected ubuntu release under test, login and run:
sudo touch /etc/krb5.keytab
sudo chmod 0600 /etc/krb5.keytab
# install nfs-common that has the bug
sudo apt install nfs-common -y
# take a snapshot of running processes
pstree > pstree.old
# purge the nfs-common package
sudo apt purge nfs-common -y
# install the new nfs-common package
sudo apt install nfs-common -y
# take a new pstree snapshot and compare with the old one
pstree > pstree.new
diff -u pstree.old pstree.new
Should be no difference.
[Where problems could occur]
Also known as "I'm doing an unconditional start in postinst, what could go wrong":
- start services that were not started with the previous package on first install
- systemd behavior change or bug and suddenly PartOf units also react to "start", instead of just "restart" and "stop" as documented
- starting services that are not configured, and start fails, breaking postinst (but we have the proverbial || true to avoid that)
[Other Info]
This fix is a bit awkward, but I think it's in line with the SRU spirit of doing the least unpredictable change, and one that is simple and can be better understood.
See the linked MP for an explanation of this fix, why it works, and other tests I did:
https://code.launchpad.net/~ahasenack/ubuntu/+source/nfs-utils/+git/nfs-utils/+merge/403288
[Original Description]
Upgrading the nfs-common debian package will not restart its services.
Specifically, the package tries to restart "nfs-utils.service", which is a "fake" service meant to coordinate all the other daemons that make up a modern NFS server. This service, however, as it is, cannot be enabled:
$ sudo systemctl enable nfs-utils.service
The unit files have no installation config (WantedBy, RequiredBy, Also, Alias
settings in the [Install] section, and DefaultInstance for template units).
This means they are not meant to be enabled using systemctl.
Possible reasons for having this kind of units are:
1) A unit may be statically enabled by being symlinked from another unit's
.wants/ or .requires/ directory.
2) A unit's purpose may be to act as a helper for some other unit which has
a requirement dependency on it.
3) A unit may be started when needed via activation (socket, path, timer,
D-Bus, udev, scripted systemctl call, ...).
4) In case of template units, the unit is meant to be enabled with some
instance name specified
Granted, d/rules of the nfs-utils package doesn't even try:
dh_systemd_enable -p nfs-common nfs-client.target
dh_systemd_enable -p nfs-kernel-server nfs-server.service
dh_installinit -pnfs-common -R
dh_systemd_start -p nfs-common --restart-after-upgrade nfs-utils.service
dh_systemd_start -p nfs-kernel-server --restart-after-upgrade nfs-server.service
We can see it tries to start and restart it, but that won't work on disabled or non-started services: deb-systemd-invoke won't do it:
# If the job is disabled and is not currently running, the job is not started or restarted.
# However, if the job is disabled but has been forced into the running state, we *do* stop
# and restart it since this is expected behaviour for the admin who forced the start.
# We don't autostart static units either.
The above can be seen while attempting a fresh install (or even upgrade) of nfs-common:
(...)
Setting up nfs-common (1:1.3.4-2.5ubuntu6) ...
Creating config file /etc/idmapd.conf with new version
Adding system user `statd' (UID 113) ...
Adding new user `statd' (UID 113) with group `nogroup' ...
Not creating home directory `/var/lib/nfs'.
Created symlink /etc/systemd/system/multi-user.target.wants/nfs-client.target → /lib/systemd/system/nfs-client.target.
Created symlink /etc/systemd/system/remote-fs.target.wants/nfs-client.target → /lib/systemd/system/nfs-client.target.
nfs-utils.service is a disabled or a static unit, not starting it.
^^^^^^^^^^^^^^^^^
$ systemctl status nfs-utils.service
● nfs-utils.service - NFS server and client services
Loaded: loaded (/lib/systemd/system/nfs-utils.service; static)
Active: inactive (dead)
This was found while testing the fix for bug #1927745. In that bug,
the affected service is rpc.gssd and it's critical that it be
restarted, but it's not happening. It will only be restarted if nfs-
utils.service is already "started".
I'm marking this bug as "high" because it prevents valid fixes from
being deployed after just upgrading a package.
To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/nfs-utils/+bug/1928259/+subscriptions
More information about the foundations-bugs
mailing list