[Bug 2033308] Please test proposed package

Thu Oct 26 13:18:39 UTC 2023

Hello Dan, or anyone else affected,

Accepted live-build into focal-proposed. The package will build now and
be available at https://launchpad.net/ubuntu/+source/live-
build/3.0~a57-1ubuntu38.20.04.3 in a few hours, and then in the
-proposed repository.

Please help us by testing this new package.  See
https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how
to enable and use -proposed.  Your feedback will aid us getting this
update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug,
mentioning the version of the package you tested, what testing has been
performed on the package and change the tag from verification-needed-
focal to verification-done-focal. If it does not fix the bug for you,
please add a comment stating that, and change the tag to verification-
failed-focal. In either case, without details of your testing we will
not be able to proceed.

Further information regarding the verification process can be found at
https://wiki.ubuntu.com/QATeam/PerformingSRUVerification .  Thank you in
advance for helping!

N.B. The updated package will be released to -updates after the bug(s)
fixed by this package have been verified and the package has been in
-proposed for a minimum of 7 days.

-- 
You received this bug notification because you are a member of Ubuntu
Foundations Bugs, which is subscribed to livecd-rootfs in Ubuntu.
https://bugs.launchpad.net/bugs/2033308

Title:
  live-build manipulation of conffiles results in them being treated as
  modified by end user

Status in live-build package in Ubuntu:
  Fix Released
Status in livecd-rootfs package in Ubuntu:
  Invalid
Status in ubuntu-advantage-tools package in Ubuntu:
  Invalid
Status in live-build source package in Xenial:
  In Progress
Status in livecd-rootfs source package in Xenial:
  Invalid
Status in ubuntu-advantage-tools source package in Xenial:
  Invalid
Status in live-build source package in Bionic:
  In Progress
Status in livecd-rootfs source package in Bionic:
  Invalid
Status in ubuntu-advantage-tools source package in Bionic:
  Invalid
Status in live-build source package in Focal:
  Fix Committed
Status in livecd-rootfs source package in Focal:
  Invalid
Status in ubuntu-advantage-tools source package in Focal:
  Invalid
Status in live-build source package in Jammy:
  Fix Committed
Status in livecd-rootfs source package in Jammy:
  Invalid
Status in ubuntu-advantage-tools source package in Jammy:
  Invalid
Status in live-build source package in Lunar:
  Fix Committed
Status in livecd-rootfs source package in Lunar:
  Invalid
Status in ubuntu-advantage-tools source package in Lunar:
  Invalid

Bug description:
  [ Impact ]

  For packages that use preferences.d/ files, live-build will remove those files
  causing the package to prompt for conffiles that it should have by default.

  This issue was first seen in ubuntu-advantage-tools, which intended to use
  conffiles in the 29 -> 29.2 transition but was prevented because of this bug.
  The current plan is to have the conffiles be added in 29.5, but since ubuntu-
  advantage-tools has an SRU exception this bug would prevent users on older than
  mantic systems being able to run 29.5+ ubuntu-advantage-tools as intended.

  To avoid removing packaged preferences.d files, the proposed patch only removes
  /etc/apt/preferences.d/* files after we backup any existing preferences.d/
  files in lb_chroot_archives install stage. Then during remove stage or
  lb_chroot_archives the backed up packaged preferences.d files are restored.

  [ Test Plan ]

  Since there are no known packages that rely on conffiles in this way, the
  easiest way to see the bug/test the patch is to use the test patch attached to
  this bug report.  This adds an additional step lb_chroot_create_preferences to
  the build process that writes a preferences file /etc/apt/preferences.d/99dummy.
  Then verify that /etc/apt/preferences.d/99dummy is present on a image built with
  live-build; it will not be there if the bug is still present.

  Individual steps to perform the test:

  # Pull unpatched live-build
  $ pull-lp-source live-build focal
  $ cd live-build-3.0~a57/
  # Add testing patch to live-build so behaviour is apparent
  $ quilt import lb_chroot_archives-preferences_test.patch
  $ quilt push

  # Create an image using unpatched live-build
  $ export UBUNTU_OLD_FASHIONED_DIR=[directory of ubuntu-bartender]
  $ export LIVECD_ROOTFS_DIR=[directory of livecd-rootfs]
  $ export LIVECD_ROOTFS_LOCAL_DIR=[directory of livecd-rootfs]
  $ export LIVE_BUILD_DIR=[directory of unpatched live-build]
  $ ubuntu-bartender --build-provider aws --livecd-rootfs-branch ubuntu/focalmaster -- --series focal --project ubuntu-cpc --image-target disk-image --subproject minimized

  # Validate the dummy preferences file was created
  $ grep 'Creating dummy preferences' [UNPATCHED]-ubuntu-bartender.log
  + Echo_message Creating dummy preferences.d files...
  + STRING=Creating dummy preferences.d files...
  P:+ printf Creating dummy preferences.d files...\n
   Creating dummy preferences.d files...

  # Download and extract the patched live-build
  # Add testing patch to live-build so behaviour is apparent
  $ quilt import lb_chroot_archives-preferences_test.patch
  $ quilt push

  # Create an image using patched live-build
  $ export UBUNTU_OLD_FASHIONED_DIR=[directory of ubuntu-bartender]
  $ export LIVECD_ROOTFS_DIR=[directory of livecd-rootfs]
  $ export LIVECD_ROOTFS_LOCAL_DIR=[directory of livecd-rootfs]
  $ export LIVE_BUILD_DIR=[directory of patched live-build]
  $ ubuntu-bartender --build-provider aws --livecd-rootfs-branch ubuntu/focalmaster -- --series focal --project ubuntu-cpc --image-target disk-image --subproject minimized

  # Validate the dummy preferences file was created
  $ grep 'Creating dummy preferences' [PATCHED]-ubuntu-bartender.log
  + Echo_message Creating dummy preferences.d files...
  + STRING=Creating dummy preferences.d files...
  P:+ printf Creating dummy preferences.d files...\n
   Creating dummy preferences.d files...

   # Mount the two images
   $ tar xvzf [UNPATCHED]-ubuntu-on-the-rocks.tar.gz
   $ mv build.output build.output-unpatched
   $ tar xvzf [PATCHED]-ubuntu-on-the-rocks.tar.gz
   $ mv build.output build.output-patched
   $ mkdir --parents --verbose ./mount-unpatched && sudo mount ./build.output-unpatched/livecd.ubuntu-cpc.ext4 ./mount-unpatched
   $ mkdir --parents --verbose ./mount-patched && sudo mount ./build.output-patched/livecd.ubuntu-cpc.ext4 ./mount-patched

   # Diff manifests (should be identical)
   $ diff build.output-patched/livecd.ubuntu-cpc.manifest build.output-unpatched/livecd.ubuntu-cpc.manifest

   # Create and diff filelists (patched version will have /etc/apt/preferences.d/99dummy; unpatched will not)
   $ (cd "./mount-unpatched" && sudo find -xdev) > "unpatched.filelist"
   $ sort unpatched.filelist > unpatched.filelist.sorted
   $ (cd "./mount-patched" && sudo find -xdev) > "patched.filelist"
   $ sort patched.filelist > patched.filelist.sorted
   $ diff patched.filelist.sorted unpatched.filelist.sorted
   230d229
  < ./etc/apt/preferences.d/99dummy

  # Clean up
  $ sudo umount ./mount-unpatched
  $ sudo umount ./mount-patched
  $ rm -r *-unpatched
  $ rm -r *-patched
  $ rm *.filelist
  $ rm *.filelist.sorted

  [ Where problems could occur ]

  Since we were not able to fully determine the history of lb_chroot_archives,
  it's possible that there was a compelling reason for wiping out
  /etc/apt/prferences.d/ in its entirety (the existing comment: probably too bold,
  needs refinment (FIXME) suggest that perhaps this was not the long term plan but
  there may have been a good reason nonetheless).  Because of this uncertainty the
  patch attempts to minimize the changes to lb_chroot_archives' existing function
  but the backup and restore do keep files that would previously have been
  deleted.

  It would be preferable for packaged apt preference files to be stored somewhere
  besides /etc but unfortunately this is how apt is written.

  Since this creates copies of existing files temporarily there may be some
  failure path where those copies are created but not properly cleaned up.  In
  that case there would be unexpected config files on the system in
  /etc/apt/preferences.d.save.  While untidy and not ideal, this state should not
  create functional problems for the user as apt will not attempt to read the
  copied files.

  If anything in lb_chroot_archives creates different files in
  /etc/apt/preferences.d with the same names as the saved versions, the saved
  version of the file will overwrite the new file.  We did not see this behavior
  in testing but it could theoretically cause problems if it did occur.  The
  secion # Check local pinning preferences in lb_chroot_archives does write to the
  directory explicitly.

  [ Other Info ]

  The original bug report, focusing on the ubuntu-advantage-tools issue has been
  retained below for reference.

  --------------------------------------------------------------------------------

  When updating ubuntu-advantage-tools from 29 -> 29.2, a conffile prompt is seen.
  This is on a system that is presumed to not have modified the relevant files.

  Setting up ubuntu-advantage-tools (29.2) ...

  Configuration file '/etc/apt/preferences.d/ubuntu-pro-esm-apps'
   ==> Deleted (by you or by a script) since installation.
   ==> Package distributor has shipped an updated version.
     What would you like to do about it ?  Your options are:
      Y or I  : install the package maintainer's version
      N or O  : keep your currently-installed version
        D     : show the differences between the versions
        Z     : start a shell to examine the situation
   The default action is to keep your current version.
  *** ubuntu-pro-esm-apps (Y/I/N/O/D/Z) [default=N] ?

  Configuration file '/etc/apt/preferences.d/ubuntu-pro-esm-infra'
   ==> Deleted (by you or by a script) since installation.
   ==> Package distributor has shipped an updated version.
     What would you like to do about it ?  Your options are:
      Y or I  : install the package maintainer's version
      N or O  : keep your currently-installed version
        D     : show the differences between the versions
        Z     : start a shell to examine the situation
   The default action is to keep your current version.
  *** ubuntu-pro-esm-infra (Y/I/N/O/D/Z) [default=N] ?

  steps to reproduce (at least today):
  lxc launch ubuntu-daily:mantic
  in container:
   apt update
   apt install ubuntu-advantage-tools

  Discovered in subiquity CI -
  https://github.com/canonical/subiquity/actions/runs/6003040269/job/16280611441?pr=1773#step:3:1080

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/live-build/+bug/2033308/+subscriptions