[Bug 1775660] [NEW] ubuntu-release-upgrader should purge obsolete packages, not just remove them

[Steve Langasek]

Public bug reported:

[Justification]
When an obsolete package leaves conffiles behind, these conffiles can sometimes interact poorly with the system and cause bugs.  Since the obsoleted package no longer exists in the targeted release, this can be difficult to resolve by applying fixes to a package.  Furthermore, removing-but-not-purging packages on upgrade means that a system which is upgraded will as a rule be inconsistent with a newly-installed system, which should always be considered a bug.

Wherever possible, we should strive to make an upgraded system look
exactly like a newly installed system, to minimize the combinatorics of
configurations that must be supported.

Examples of cases where this has been a problem in practice:

LP: #1770965
LP: #1749199

[Test case]
1. Do an upgrade from xenial with the release ubuntu-release-upgrader.  Accept the option to remove obsolete packages.
2. Capture the list of packages that are in state 'rc' afterwards.
3. Do an equivalent upgrade with the -proposed ubuntu-release-upgrader, including removal of obsolete packages.
4. Confirm that the upgrade completes successfully.
5. Capture the list of packages that are in state 'rc' afterwards.
6. Confirm that there are fewer packages in state 'rc' in the second upgrade than the first.
7. Repeat all of the above with an upgrade from artful.

[Regression potential]
As a consequence of this, we will also be running a lot more postrm scripts with the 'purge' option on users desktops, so any bugs in those maintainer scripts will now be apparent as part of a release upgrade where before they were latent.

** Affects: ubuntu-release-upgrader (Ubuntu)
     Importance: High
         Status: Triaged

** Affects: ubuntu-release-upgrader (Ubuntu Bionic)
     Importance: High
         Status: Triaged

** Also affects: ubuntu-release-upgrader (Ubuntu Bionic)
   Importance: Undecided
       Status: New

** Changed in: ubuntu-release-upgrader (Ubuntu)
   Importance: Undecided => High

** Changed in: ubuntu-release-upgrader (Ubuntu Bionic)
   Importance: Undecided => High

** Changed in: ubuntu-release-upgrader (Ubuntu)
       Status: New => Triaged

** Changed in: ubuntu-release-upgrader (Ubuntu Bionic)
       Status: New => Triaged

** Changed in: ubuntu-release-upgrader (Ubuntu Bionic)
    Milestone: None => ubuntu-18.04.1

-- 
You received this bug notification because you are a member of Ubuntu
Foundations Bugs, which is subscribed to ubuntu-release-upgrader in
Ubuntu.
https://bugs.launchpad.net/bugs/1775660

Title:
  ubuntu-release-upgrader should purge obsolete packages, not just
  remove them

Status in ubuntu-release-upgrader package in Ubuntu:
  Triaged
Status in ubuntu-release-upgrader source package in Bionic:
  Triaged

Bug description:
  [Justification]
  When an obsolete package leaves conffiles behind, these conffiles can sometimes interact poorly with the system and cause bugs.  Since the obsoleted package no longer exists in the targeted release, this can be difficult to resolve by applying fixes to a package.  Furthermore, removing-but-not-purging packages on upgrade means that a system which is upgraded will as a rule be inconsistent with a newly-installed system, which should always be considered a bug.

  Wherever possible, we should strive to make an upgraded system look
  exactly like a newly installed system, to minimize the combinatorics
  of configurations that must be supported.

  Examples of cases where this has been a problem in practice:

  LP: #1770965
  LP: #1749199

  [Test case]
  1. Do an upgrade from xenial with the release ubuntu-release-upgrader.  Accept the option to remove obsolete packages.
  2. Capture the list of packages that are in state 'rc' afterwards.
  3. Do an equivalent upgrade with the -proposed ubuntu-release-upgrader, including removal of obsolete packages.
  4. Confirm that the upgrade completes successfully.
  5. Capture the list of packages that are in state 'rc' afterwards.
  6. Confirm that there are fewer packages in state 'rc' in the second upgrade than the first.
  7. Repeat all of the above with an upgrade from artful.

  [Regression potential]
  As a consequence of this, we will also be running a lot more postrm scripts with the 'purge' option on users desktops, so any bugs in those maintainer scripts will now be apparent as part of a release upgrade where before they were latent.

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/ubuntu-release-upgrader/+bug/1775660/+subscriptions