[Bug 1260041] Re: Unattended-Upgrades crashes with a dpkg error

[Balint Reczey]

** Description changed:

+ [Impact]
+ 
+  * Unattended-upgrades fails upgrading packages in the middle of a dpkg/apt call due to a parallel process acquiring apt or dpkg lock while u-u is running, but u-u running apt/dpkg anyway without successfully re-acquiring the lock.
+  * The fix is stopping processing updates when the lock is detected to be lost.
+ 
+ [Test Case]
+ 
+  * Set up a system with packages (> 30) to be upgraded:
+ $ lxc launch ubuntu:b36ec647e374 uu-lock-bail
+ 
+  * Set up two shells to run commands in parallel
+  * In shell "A" run sudo apt update && sudo unattended-upgrade --dry-run --verbose --debug
+  * After u-u started installing the updates run the following command in shell "B":
+ $  while sleep 0.1; do python3 -c 'import apt, apt_pkg, time; print(apt_pkg.pkgsystem_lock()); time.sleep(2)' ; done
+ 
+ * With not fixed u-u observe the following exception repeated in shell "B", then True being printed when the lock is successfully stolen from u-u:
+ ...
+ Traceback (most recent call last):
+   File "<string>", line 1, in <module>
+ apt_pkg.Error: E:Could not get lock /var/lib/dpkg/lock - open (11: Resource temporarily unavailable), E:Unable to lock the administration directory (/var/lib/dpkg/), is another process using it?
+ Traceback (most recent call last):
+   File "<string>", line 1, in <module>
+ apt_pkg.Error: E:Could not get lock /var/lib/dpkg/lock - open (11: Resource temporarily unavailable), E:Unable to lock the administration directory (/var/lib/dpkg/), is another process using it?
+ True
+ Traceback (most recent call last):
+   File "<string>", line 1, in <module>
+ apt_pkg.Error: E:Could not get lock /var/lib/dpkg/lock - open (11: Resource temporarily unavailable), E:Unable to lock the administration directory (/var/lib/dpkg/), is another process using it?
+ True
+ True
+ True
+ True
+ ...
+ 
+ Buggy u-u continues operation even after the lock is stolen.
+ 
+ Fixed u-u stops operation with a printout similar to the following when it detects the lock to be stolen:
+ ...
+ Writing dpkg log to /var/log/unattended-upgrades/unattended-upgrades-dpkg.log
+ /usr/bin/dpkg --status-fd 9 --no-triggers --unpack --auto-deconfigure /var/cache/apt/archives/distro-info-data_0.37ubuntu0.1_all.deb 
+ /usr/bin/dpkg --status-fd 9 --configure --pending 
+ Exception happened during upgrade.
+ Traceback (most recent call last):
+   File "./unattended-upgrade", line 539, in cache_commit
+     res = cache.commit(install_progress=iprogress)
+   File "./unattended-upgrade", line 360, in __exit__
+     apt_pkg.pkgsystem_lock()
+ apt_pkg.Error: E:Could not get lock /var/lib/dpkg/lock - open (11: Resource temporarily unavailable), E:Unable to lock the administration directory (/var/lib/dpkg/), is another process using it?
+ Exception happened during upgrade.
+ NoneType: None
+ Installing the upgrades failed!
+ error message: E:Could not get lock /var/lib/dpkg/lock - open (11: Resource temporarily unavailable), E:Unable to lock the administration directory (/var/lib/dpkg/), is another process using it?
+ dpkg returned a error! See /var/log/unattended-upgrades/unattended-upgrades-dpkg.log for details
+ ...
+ 
+ [Regression Potential]
+ 
+  * As a consequence of the fix unattended-upgrade stops installing
+ security-updates when an other dpkg frontend's parallel activity is
+ detected. This is intentional to prevent parallel execution of dpkg and
+ potentially leaving the system in an inconsistent state. The full fix
+ will be implemented in LP: #1781169 and changing u-u to use the new
+ locking mechanism, and then u-u will never lose the lock.
+ 
+ [Other Info]
+ 
+ I believe parallel dpkg calls cause many breakages reported against u-u
+ and other packages listing breaking maintainer scripts.
+ 
+ [Original Bug Text]
+ 
  As per https://bugs.launchpad.net/ubuntu/+source/unattended-
  upgrades/+bug/1052360/comments/17, I am filing a new bug with the
  specific error I am running into. This has been going on for quite a
  while now.
  
  Also, my log files still show up as I described in
  https://bugs.launchpad.net/ubuntu/+source/unattended-
  upgrades/+bug/1052360/comments/14.
  
  ProblemType: Bug
  DistroRelease: Ubuntu 13.10
  Package: unattended-upgrades 0.79.3ubuntu8
  ProcVersionSignature: Ubuntu 3.11.0-14.21-generic 3.11.7
  Uname: Linux 3.11.0-14-generic x86_64
  NonfreeKernelModules: fglrx
  ApportVersion: 2.12.5-0ubuntu2.1
  Architecture: amd64
  Date: Wed Dec 11 11:07:09 2013
  MarkForUpload: True
  PackageArchitecture: all
  SourcePackage: unattended-upgrades
  UpgradeStatus: Upgraded to saucy on 2013-10-18 (54 days ago)

-- 
You received this bug notification because you are a member of Ubuntu
Foundations Bugs, which is subscribed to unattended-upgrades in Ubuntu.
https://bugs.launchpad.net/bugs/1260041

Title:
  Unattended-Upgrades crashes with a dpkg error

Status in unattended-upgrades package in Ubuntu:
  New

Bug description:
  [Impact]

   * Unattended-upgrades fails upgrading packages in the middle of a dpkg/apt call due to a parallel process acquiring apt or dpkg lock while u-u is running, but u-u running apt/dpkg anyway without successfully re-acquiring the lock.
   * The fix is stopping processing updates when the lock is detected to be lost.

  [Test Case]

   * Set up a system with packages (> 30) to be upgraded:
  $ lxc launch ubuntu:b36ec647e374 uu-lock-bail

   * Set up two shells to run commands in parallel
   * In shell "A" run sudo apt update && sudo unattended-upgrade --dry-run --verbose --debug
   * After u-u started installing the updates run the following command in shell "B":
  $  while sleep 0.1; do python3 -c 'import apt, apt_pkg, time; print(apt_pkg.pkgsystem_lock()); time.sleep(2)' ; done

  * With not fixed u-u observe the following exception repeated in shell "B", then True being printed when the lock is successfully stolen from u-u:
  ...
  Traceback (most recent call last):
    File "<string>", line 1, in <module>
  apt_pkg.Error: E:Could not get lock /var/lib/dpkg/lock - open (11: Resource temporarily unavailable), E:Unable to lock the administration directory (/var/lib/dpkg/), is another process using it?
  Traceback (most recent call last):
    File "<string>", line 1, in <module>
  apt_pkg.Error: E:Could not get lock /var/lib/dpkg/lock - open (11: Resource temporarily unavailable), E:Unable to lock the administration directory (/var/lib/dpkg/), is another process using it?
  True
  Traceback (most recent call last):
    File "<string>", line 1, in <module>
  apt_pkg.Error: E:Could not get lock /var/lib/dpkg/lock - open (11: Resource temporarily unavailable), E:Unable to lock the administration directory (/var/lib/dpkg/), is another process using it?
  True
  True
  True
  True
  ...

  Buggy u-u continues operation even after the lock is stolen.

  Fixed u-u stops operation with a printout similar to the following when it detects the lock to be stolen:
  ...
  Writing dpkg log to /var/log/unattended-upgrades/unattended-upgrades-dpkg.log
  /usr/bin/dpkg --status-fd 9 --no-triggers --unpack --auto-deconfigure /var/cache/apt/archives/distro-info-data_0.37ubuntu0.1_all.deb 
  /usr/bin/dpkg --status-fd 9 --configure --pending 
  Exception happened during upgrade.
  Traceback (most recent call last):
    File "./unattended-upgrade", line 539, in cache_commit
      res = cache.commit(install_progress=iprogress)
    File "./unattended-upgrade", line 360, in __exit__
      apt_pkg.pkgsystem_lock()
  apt_pkg.Error: E:Could not get lock /var/lib/dpkg/lock - open (11: Resource temporarily unavailable), E:Unable to lock the administration directory (/var/lib/dpkg/), is another process using it?
  Exception happened during upgrade.
  NoneType: None
  Installing the upgrades failed!
  error message: E:Could not get lock /var/lib/dpkg/lock - open (11: Resource temporarily unavailable), E:Unable to lock the administration directory (/var/lib/dpkg/), is another process using it?
  dpkg returned a error! See /var/log/unattended-upgrades/unattended-upgrades-dpkg.log for details
  ...

  [Regression Potential]

   * As a consequence of the fix unattended-upgrade stops installing
  security-updates when an other dpkg frontend's parallel activity is
  detected. This is intentional to prevent parallel execution of dpkg
  and potentially leaving the system in an inconsistent state. The full
  fix will be implemented in LP: #1781169 and changing u-u to use the
  new locking mechanism, and then u-u will never lose the lock.

  [Other Info]

  I believe parallel dpkg calls cause many breakages reported against
  u-u and other packages listing breaking maintainer scripts.

  [Original Bug Text]

  As per https://bugs.launchpad.net/ubuntu/+source/unattended-
  upgrades/+bug/1052360/comments/17, I am filing a new bug with the
  specific error I am running into. This has been going on for quite a
  while now.

  Also, my log files still show up as I described in
  https://bugs.launchpad.net/ubuntu/+source/unattended-
  upgrades/+bug/1052360/comments/14.

  ProblemType: Bug
  DistroRelease: Ubuntu 13.10
  Package: unattended-upgrades 0.79.3ubuntu8
  ProcVersionSignature: Ubuntu 3.11.0-14.21-generic 3.11.7
  Uname: Linux 3.11.0-14-generic x86_64
  NonfreeKernelModules: fglrx
  ApportVersion: 2.12.5-0ubuntu2.1
  Architecture: amd64
  Date: Wed Dec 11 11:07:09 2013
  MarkForUpload: True
  PackageArchitecture: all
  SourcePackage: unattended-upgrades
  UpgradeStatus: Upgraded to saucy on 2013-10-18 (54 days ago)

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/unattended-upgrades/+bug/1260041/+subscriptions