[Bug 1654600] Re: unattended-upgrade-shutdown hangs when /var is a separate filesystem

[Louis Bouchard]

Hello,

This logic is correct. apt_pkg.get_lock(options.lock_file) is trying to
acquire a lockfile (/var/run/unattended-upgrades.lock) which is only
possible if no unattended-upgrade is already running. If it can acquire
the lock, it means that NO unattended-upgrade is running, hence shutdown
may proceed.

 while True:                                                                                                    
        res = apt_pkg.get_lock(options.lock_file)                                                                  
        logging.debug("get_lock returned %i" % res)                                                                
        # exit here if there is no lock                                                                            
        if res > 0:                                                                                                
            logging.debug("lock not taken")                                                                        
            break                                                                                                  
        lock_was_taken = True

The problem is that apt_pkg.get_lock(options.lock_file) will also return
-1 if the path to the lock file doesn't exist, which is the case when
/var is unmounted, hence /var/run is no longer present (/var/run is a
symlink to /run).

I'm working on identifying the proper systemd order to make sure that
/var/run is still accessible.

-- 
You received this bug notification because you are a member of Ubuntu
Foundations Bugs, which is subscribed to unattended-upgrades in Ubuntu.
https://bugs.launchpad.net/bugs/1654600

Title:
  unattended-upgrade-shutdown hangs when /var is a separate filesystem

Status in unattended-upgrades package in Ubuntu:
  Confirmed
Status in unattended-upgrades source package in Xenial:
  Confirmed
Status in unattended-upgrades source package in Yakkety:
  New

Bug description:
  The systemd unit file unattended-upgrades.service is used to stop a running unattended-upgrade
  process during shutdown. This unit file is running together with all filesystem
  unmount services.

  The unattended-upgrades service checks if the lockfile for unattended-upgrade
  (in /var/run) exists, and if it does, there is an unattended-upgrade in progress
  and the service will wait until it finishes (and therefore automatically wait at
  shutdown).

  However, if /var is a separate filesystem, it will get unmounted even though /var/run
  is a tmpfs that's still mounted on top of the /var/run directory in the /var filesystem.
  The unattended-upgrade script will fail to find lockfile, sleeps for 5 seconds, and
  tries to check the lockfile again. After 10 minutes (the default timeout), it will finally
  exit and the system will continue shutdown.

  The problem is the error handling in /usr/share/unattended-upgrades/unattended-upgrade-shutdown
  where it tries to lock itself:

      while True:
          res = apt_pkg.get_lock(options.lock_file)
          logging.debug("get_lock returned %i" % res)
          # exit here if there is no lock
          if res > 0:
              logging.debug("lock not taken")
              break
          lock_was_taken = True

  The function apt_pkg.get_lock() either returns a file descriptor, or -1 on an error.
  File descriptors are just C file descriptors, so they are always positive integers.
  The code should check the result to be negative, not positive. I have attached a patch
  to reverse the logic.

  Additional information:

  1)
  Description:	Ubuntu 16.04.1 LTS
  Release:	16.04

  2)
  unattended-upgrades:
    Installed: 0.90ubuntu0.3
    Candidate: 0.90ubuntu0.3
    Version table:
   *** 0.90ubuntu0.3 500
          500 http://nl.archive.ubuntu.com/ubuntu xenial-updates/main amd64 Packages
          500 http://nl.archive.ubuntu.com/ubuntu xenial-updates/main i386 Packages
          100 /var/lib/dpkg/status
       0.90 500
          500 http://nl.archive.ubuntu.com/ubuntu xenial/main amd64 Packages
          500 http://nl.archive.ubuntu.com/ubuntu xenial/main i386 Packages
  3)
  Fast reboot
  4)
  Very slow reboot (after a 10 minutes timeout)

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/unattended-upgrades/+bug/1654600/+subscriptions