[Bug 1823410] Re: utils.py's url_downloadable doesn't support https

[Brian Murray]

I tested a release upgrade from Bionic to Cosmic with https entries in
my sources.list file. With the version of the release upgrader in the
-updates pocket I was not allowed to upgrade however when I tried to
upgrade using 'do-release-upgrade -p' the upgrade was successfully
calculated. Here's an excerpt from /var/log/dist-upgrade/main.log:


2019-04-15 09:15:10,252 DEBUG examining: 'deb https://ubuntu.osuosl.org/ubuntu bionic main restricted universe multiverse'
2019-04-15 09:15:10,253 DEBUG found apt-cacher/apt-torrent style uri https://ubuntu.osuosl.org/ubuntu
2019-04-15 09:15:10,253 DEBUG verifySourcesListEntry: deb https://ubuntu.osuosl.org/ubuntu cosmic main restricted universe multiverse
2019-04-15 09:15:10,253 DEBUG url_downloadable: https://ubuntu.osuosl.org/ubuntu/dists/cosmic/Release
2019-04-15 09:15:10,253 DEBUG s='https' n='ubuntu.osuosl.org' p='/ubuntu/dists/cosmic/Release' q='' f=''
2019-04-15 09:15:10,623 DEBUG entry 'deb https://ubuntu.osuosl.org/ubuntu cosmic main restricted universe multiverse' updated to new dist

The "apt-cacher/apt-torrent" message is a bit strange but it worked.

-- 
You received this bug notification because you are a member of Ubuntu
Foundations Bugs, which is subscribed to update-manager in Ubuntu.
https://bugs.launchpad.net/bugs/1823410

Title:
  utils.py's url_downloadable doesn't support https

Status in ubuntu-release-upgrader package in Ubuntu:
  Fix Released
Status in update-manager package in Ubuntu:
  Fix Released
Status in ubuntu-release-upgrader source package in Bionic:
  Fix Committed
Status in update-manager source package in Bionic:
  Fix Committed
Status in ubuntu-release-upgrader source package in Cosmic:
  Fix Committed
Status in update-manager source package in Cosmic:
  Fix Committed

Bug description:
  [Impact]
  In the interest of improving release upgrades for users of PPAs a check was added to ubuntu-release-upgrader to see if the release to which the system is being upgraded is provided by a sources list entry and if it is not that entry is disabled. This check uses the function url_downloadable which is provided by update-manager's utils.py. Come to find out the url_downloadable check only supports ftp and http, not https. This can cause a situation where upgrades to the next release of Ubuntu fail if the system's mirror of the Ubuntu archive is using https. This is mentioned here - https://bugs.launchpad.net/ubuntu/+source/ubuntu-release-upgrader/+bug/1807043/comments/10.

  While the bug manifests itself in ubuntu-release-upgrader it needs
  fixing in update-manager and will require a rebuild of ubuntu-release-
  upgrader which has a symlink to utils.py which is shipped in the dist-
  upgrader tarball.

  [Test Case]
  update-manager - only ubuntu-release-upgrader uses url_downloadable but I added a test to update-manager to ensure that https works, the test may be skipped though if the https test site is not available so keep an eye out for that.

  ubuntu-release-upgrader
  1) On a xenial or bionic system edit your sources.list file so that you are using an https mirror of the archive.
  2) try to upgrade to the next release of ubuntu using 'do-release-upgrade'

  With the version of ubuntu-release-upgrader in the archive you will
  not be allowed to upgrade because ubuntu-minimal will not be found.
  With the version in -proposed, accessed by using 'do-release-upgrade
  -p' you should be able to upgrade to the next release of Ubuntu.

  [Regression Potential]
  There isn't much chance of a regression as we are just adding https to the list of schemes that we will test with urlopen.

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/ubuntu-release-upgrader/+bug/1823410/+subscriptions