[Bug 1822886] Re: universe missing after bionic->cosmic do-release-upgrade

[Brian Murray]

I followed the steps in the test case when upgrading from bionic to
cosmic and it passed. In the main.log file of the upgrade using 'do-
release-upgrade -p' we can see url_downloadable is called only once and
that the universe entry is still enabled.

2019-04-15 10:13:22,505 INFO Using config files '['./DistUpgrade.cfg']'
2019-04-15 10:13:22,505 INFO uname information: 'Linux clean-bionic-amd64 4.15.0-47-generic #50-Ubuntu SMP Wed Mar 13 10:44:52 UTC 2019 x86_64'
2019-04-15 10:13:22,517 INFO apt version: '1.6.10'
2019-04-15 10:13:22,517 INFO python version: '3.6.7 (default, Oct 22 2018, 11:32:17)
[GCC 8.2.0]'
2019-04-15 10:13:22,517 INFO release-upgrader version '18.10.11.6' started
...
2019-04-15 10:13:47,165 DEBUG examining: 'deb http://ubuntu.osuosl.org/ubuntu bionic main restricted multiverse'
2019-04-15 10:13:47,165 DEBUG verifySourcesListEntry: deb http://ubuntu.osuosl.org/ubuntu cosmic main restricted multiverse
2019-04-15 10:13:47,166 DEBUG url_downloadable: http://ubuntu.osuosl.org/ubuntu/dists/cosmic/Release
2019-04-15 10:13:47,166 DEBUG s='http' n='ubuntu.osuosl.org' p='/ubuntu/dists/cosmic/Release' q='' f=''
2019-04-15 10:13:47,330 DEBUG entry 'deb http://ubuntu.osuosl.org/ubuntu cosmic main restricted multiverse' updated to new dist
2019-04-15 10:13:47,331 DEBUG examining: 'deb http://ubuntu.osuosl.org/ubuntu bionic-updates main restricted multiverse'
2019-04-15 10:13:47,331 DEBUG entry 'deb http://ubuntu.osuosl.org/ubuntu cosmic-updates main restricted multiverse' updated to new dist
2019-04-15 10:13:47,331 DEBUG examining: 'deb http://ubuntu.osuosl.org/ubuntu bionic-security main restricted multiverse'
2019-04-15 10:13:47,332 DEBUG entry 'deb http://ubuntu.osuosl.org/ubuntu cosmic-security main restricted multiverse' updated to new dist
2019-04-15 10:13:47,332 DEBUG examining: 'deb http://ubuntu.osuosl.org/ubuntu bionic universe'
2019-04-15 10:13:47,332 DEBUG entry 'deb http://ubuntu.osuosl.org/ubuntu cosmic universe' updated to new dist

Setting to verification-done for bionic and cosmic.

** Tags removed: verification-needed verification-needed-bionic verification-needed-cosmic
** Tags added: verification-done verification-done-bionic verification-done-cosmic

-- 
You received this bug notification because you are a member of Ubuntu
Foundations Bugs, which is subscribed to ubuntu-release-upgrader in
Ubuntu.
https://bugs.launchpad.net/bugs/1822886

Title:
  universe missing after bionic->cosmic do-release-upgrade

Status in ubuntu-release-upgrader package in Ubuntu:
  Fix Released
Status in ubuntu-release-upgrader source package in Bionic:
  Fix Committed
Status in ubuntu-release-upgrader source package in Cosmic:
  Fix Committed

Bug description:
  [Impact]
  If there is a network hiccup at the right moment it's possible that some sources.list entries will be disabled when a release upgrade is calculated and then people could upgrade with surprising results e.g. universe being missing.

  [Test Case]
  This is somewhat convoluted as we need to create the network error only when a specific line is is being tested by the dist-upgrader.

  1) Use a system on xenial or bionic and ensure prompt is set appropriately (lts for xenial, normal for bionic)
  2) Edit your /etc/apt/sources.list file so that universe is a separate line from main, restricted, and multiverse
  3) Run do-release-upgrade
  4) Cancel the release upgrade
  5) cd to /tmp/ubuntu-release-upgrader-$tmpname
  6) edit DistUpgradeController.py with the attached patch which will cause the url check for the universe line to fail
  7) run 'sudo ./$release-upgrading-to'
  8) wait until you receive the message regarding quantities of packages to install, upgrade, etc
  9) look at /etc/apt/sources.list file and notice that the lines with universe are disabled

  With the release-upgrader from -proposed you'll need to follow the
  same steps and you'll notice that the lines with universe are not
  disabled.

  [Regression Potential]
  Because we are checking each unique uri in sources.list only one time its possible that if the first check fails then the upgrade will fail to be calculated. That is better than upgrading and having components disabled though.

  [Original Description]

  I took a bionic system and stepped through bionic->cosmic->disco via
  do-release-upgrade.

  Upon finishing I noticed that universe was "gone" and all packages
  from universe were removed from the system. Based on the files on my
  system, it looks like it happened during the bionic->cosmic upgrade.
  You'll see in the sources.list that cosmic-updates universe pocket is
  that but not cosmic universe.

  ProblemType: BugDistroRelease: Ubuntu 19.04
  Package: ubuntu-release-upgrader-core 1:19.04.13
  ProcVersionSignature: Ubuntu 5.0.0-8.9-generic 5.0.1
  Uname: Linux 5.0.0-8-generic x86_64
  NonfreeKernelModules: zfs zunicode zavl icp zcommon znvpair
  ApportVersion: 2.20.10-0ubuntu23
  Architecture: amd64
  CrashDB: ubuntu
  CurrentDesktop: ubuntu:GNOME
  Date: Tue Apr  2 16:04:34 2019
  InstallationDate: Installed on 2018-07-24 (252 days ago)
  InstallationMedia: Ubuntu 18.04.1 LTS "Bionic Beaver" - Release amd64 (20180724)
  PackageArchitecture: all
  ProcEnviron:
   TERM=xterm
   PATH=(custom, no user)
   XDG_RUNTIME_DIR=<set>
   LANG=en_US.UTF-8
   SHELL=/bin/bashSourcePackage: ubuntu-release-upgrader
  Symptom: ubuntu-release-upgrader
  UpgradeStatus: Upgraded to disco on 2019-04-02 (0 days ago)
  VarLogDistupgradeXorgFixuplog:
   INFO:root:/usr/bin/do-release-upgrade running
   INFO:root:No xorg.conf, exiting
  mtime.conffile..etc.update-manager.release-upgrades: 2019-04-02T08:18:06.798258

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/ubuntu-release-upgrader/+bug/1822886/+subscriptions