[Bug 2070432] Re: aptsources: Legitimate entries are set to invalid when options are present

[Nick Rosbrook]

** Tags added: rls-pp-incoming

-- 
You received this bug notification because you are a member of Ubuntu
Foundations Bugs, which is subscribed to python-apt in Ubuntu.
https://bugs.launchpad.net/bugs/2070432

Title:
  aptsources: Legitimate entries are set to invalid when options are
  present

Status in python-apt package in Ubuntu:
  New

Bug description:
  [Impact]

  Ubuntu Release Upgrader uses python-apt for apt operations, including
  parsing and making sure the sources.list file is sensible.

  Options are placed in square brackets [] after the initial deb as key
  value pairs.

  e.g.

  deb [lang=en] http://archive.ubuntu.com/ubuntu focal main restricted

  sets the lang option.

  If you look at man sources.list(5), there are a number of options supported:
  * arch
  * lang
  * target
  * pdiffs
  * by-hash
  * allow-insecure
  * trusted
  * signed-by
  * check-valid-until
  * valid-until-min
  * check-date
  * date-max-future
  * inrelease-path
  * snapshot

  Currently, only arch and trusted are the only ones implemented:

  https://git.launchpad.net/ubuntu/+source/python-
  apt/tree/aptsources/sourceslist.py?h=applied/ubuntu/focal-updates#n202

      def parse(self, line):
  ...
          if pieces[1].strip()[0] == "[":
              options = pieces.pop(1).strip("[]").split()
              for option in options:
                  try:
                      key, value = option.split("=", 1)
                  except Exception:
                      self.invalid = True
                  else:
                      if key == "arch":
                          self.architectures = value.split(",")
                      elif key == "trusted":
                          self.trusted = apt_pkg.string_to_bool(value)
                      else:
                          self.invalid = True

  Usage of any others causes entries to be set invalid, and Ubuntu
  Release Upgrader then requires you to set new entries before it can
  proceed.

  This is a problem when used with an internal aptly mirror, using
  signed-by.

  The error seen on do-release-upgrade is:

  No valid mirror found

  While scanning your repository information no mirror entry for the 
  upgrade was found. This can happen if you run an internal mirror or 
  if the mirror information is out of date. 

  Do you want to rewrite your 'sources.list' file anyway? If you choose 
  'Yes' here it will update all 'focal' to 'jammy' entries. 
  If you select 'No' the upgrade will cancel.

  [Testcase]

  Simply set any supported option at all to sources.list:

  e.g.

  $ sudo sed -i "s/deb/deb [lang=en]/g" /etc/apt/sources.list

  and run:

  #!/bin/python3

  from aptsources.sourceslist import SourcesList, SourceEntry

  source = SourcesList()
  for entry in source.list:
      if (entry.invalid == False and entry.disabled == False):
          print(entry.uri + ' ' + entry.dist + ' ' + entry.comps[0])
          
  You will get no output at all.

  We would expect output like:

  http://archive.ubuntu.com/ubuntu focal main
  http://archive.ubuntu.com/ubuntu focal-updates main
  http://archive.ubuntu.com/ubuntu focal universe
  http://archive.ubuntu.com/ubuntu focal-updates universe
  http://archive.ubuntu.com/ubuntu focal multiverse
  http://archive.ubuntu.com/ubuntu focal-updates multiverse
  http://archive.ubuntu.com/ubuntu focal-backports main
  http://security.ubuntu.com/ubuntu focal-security main
  http://security.ubuntu.com/ubuntu focal-security universe
  http://security.ubuntu.com/ubuntu focal-security multiverse

  [Other info]

  This seems to have been the case since 2011, and no one has really hit
  it.

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/python-apt/+bug/2070432/+subscriptions