[Bug 1893889] Re: unattended-upgrade of nova-common failure due to conffile prompt

[Trent Lloyd]

** Attachment added: "all unattended-upgrades and dpkg logs"
   https://bugs.launchpad.net/ubuntu/+source/unattended-upgrades/+bug/1893889/+attachment/5406806/+files/lp1893889-logs.tar.gz

-- 
You received this bug notification because you are a member of Ubuntu
Foundations Bugs, which is subscribed to unattended-upgrades in Ubuntu.
https://bugs.launchpad.net/bugs/1893889

Title:
  unattended-upgrade of nova-common failure due to conffile prompt

Status in unattended-upgrades package in Ubuntu:
  New

Bug description:
  unattended-upgrades attempted to upgrade nova from 2:17.0.9-0ubuntu1
  to 2:17.0.10-0ubuntu2.1 (bionic-security), however nova-common
  contains a modified conffile (/etc/nova/nova.conf) which prompts
  during upgrade and leaves apt/dpkg in a permanent error state
  requiring manual intervention. It also prevents other automated apt
  install operations from working while in this state.

  I understand that this conffile prompt is a generally known problem
  and that unattended-upgrades specifically attempts to skip upgrades
  that have such a conffile prompt, however that did not work on this
  case. I am filing this bug to try and identify and resolve the cause
  and this affected multiple systems in an Ubuntu OpenStack deployment.

  rbalint advised that this is very likely a more complex interaction with the exact upgrades that were being staged at the time and hence more logs would be needed, indeed attempting to reproduce this very simply with a downgrade of nova packages to 2:17.0.0-0ubuntu1 results in it being skipped, as expected:
  root at juju-c21ec6-bionic-nova-7:/home/ubuntu# unattended-upgrade
  Package nova-common has conffile prompt and needs to be upgraded manually

  And from the unattended-upgrades log we can see that 179 packages in
  total were scheduled to upgrade together during this run.

  Attaching the following logs files:
  /var/log/unattended-upgrades/*
  /var/log/dpkg*
  dpkg_-l (As at 2020-04-27 16:22, the same time period as the unattended-upgrades logs, but the dpkg.log* files were taken later but also cover the full time period from before 2019-12-28 and after 2020-04-27).

  The first instance of the failure is in unattended-upgrades.log.4.gz Line 161
  "2019-12-28 06:15:29,837 Packages that will be upgraded: amd64-microcode... [truncated, 179 packages total]"

  That relates to the output in unattended-upgrades-dpkg.log.4.gz Line
  791 "Log started: 2019-12-28  06:25:56"

  Which relates to the output of dpkg.log.6.gz Line 392
  "2019-12-28 06:25:56 upgrade nova-compute-kvm:all 2:17.0.9-0ubuntu1 2:17.0.10-0ubuntu2.1"

  It fails many times after that as anytime you attempt to install a
  package, it attempts to configure nova.conf again and exits with an
  error again. But that is the original failure. But note that various
  package upgrades happened by unattended-upgrades (and possibly other
  sources) in the intervening 4 months and so I guess reproducing the
  situation may require reverse engineering the original package list
  from the dpkg logs. I have not currently attempted to do that with the
  hopes intimate knowledge of the unattended-upgrades code and logs will
  make that process faster.

  A full sosreport from the system is available if more information is
  required that will include other log files, and various other command
  outputs. It is not uploaded initially for privacy.

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/unattended-upgrades/+bug/1893889/+subscriptions