[Bug 1309447] Re: Unicode decode error during upgrade to 14.04 if sources.list contains non-ascii characters and locale is non-US

[Brian Murray]

Yes, we would need to update the Trusty version of ubuntu-release-
upgrader for people to get the newest version of apt-clone. However, it
is only necessary for people upgrading from Saucy to Trusty.  I went
looking for failures like this for people upgrading from Saucy to Trusty
in the Ubuntu Error Tracker and was unable to find any because we
stopped accepting crash reports from Saucy. Subsequently, we do not
really have an idea of how many people are still affected by this
specific bug report and therefore whether or not it is worth updating
ubuntu-release-upgrader.

-- 
You received this bug notification because you are a member of Ubuntu
Foundations Bugs, which is subscribed to apt-clone in Ubuntu.
https://bugs.launchpad.net/bugs/1309447

Title:
  Unicode decode error during upgrade to 14.04 if sources.list contains
  non-ascii characters and locale is non-US

Status in “apt-clone” package in Ubuntu:
  Fix Released
Status in “ubuntu-release-upgrader” package in Ubuntu:
  Fix Released
Status in “apt-clone” source package in Trusty:
  Fix Released
Status in “ubuntu-release-upgrader” source package in Trusty:
  In Progress

Bug description:
  TEST CASE:
  1. On Saucy or Precise add non-ascii characters to the file /etc/apt/sources.list like:

  #  Français accentué
  deb http://archive.ubuntu.com/ubuntu saucy main restricted universe multiverse
  deb http://archive.ubuntu.com/ubuntu saucy-updates main restricted universe multiverse
  deb http://security.ubuntu.com/ubuntu saucy-security main restricted universe multiverse

  2. Upgrade with a non-us locale
  LANG=fr_FR.utf8 do-release-upgrade

  ACTUAL RESULT
  This crash:
  Traceback (most recent call last):
    File "/tmp/ubuntu-release-upgrader-ui8dvi/trusty", line 10, in <module>
      sys.exit(main())
    File "/tmp/ubuntu-release-upgrader-ui8dvi/DistUpgrade/DistUpgradeMain.py", line 240, in main
      save_system_state(logdir)
    File "/tmp/ubuntu-release-upgrader-ui8dvi/DistUpgrade/DistUpgradeMain.py", line 133, in save_system_state
      scrub_sources=True)
    File "/tmp/ubuntu-release-upgrader-ui8dvi/DistUpgrade/apt_clone.py", line 150, in save_state
      self._write_state_sources_list(tar, scrub_sources)
    File "/tmp/ubuntu-release-upgrader-ui8dvi/DistUpgrade/apt_clone.py", line 231, in _write_state_sources_list
      "./etc/apt/sources.list")
    File "/tmp/ubuntu-release-upgrader-ui8dvi/DistUpgrade/apt_clone.py", line 251, in _add_file_to_tar_with_password_check
      for line in f.readlines():
    File "/usr/lib/python2.7/encodings/ascii.py", line 26, in decode
      return codecs.ascii_decode(input, self.errors)[0]
  UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 7: ordinal not in range(128)

  == Original report ==
  do-release-upgrade fails as below. I am using Polish utf-8 locale, so I suspect something called reports error in Polish with non-ascii characters...

  File "/tmp/ubuntu-release-upgrader-j2xvu5/trusty", line 10, in
  <module>
  sys.exit(main())

  File
  "/tmp/ubuntu-release-upgrader-j2xvu5/DistUpgrade/DistUpgradeMain.py",
  line 240, in main
  save_system_state(logdir)

  File
  "/tmp/ubuntu-release-upgrader-j2xvu5/DistUpgrade/DistUpgradeMain.py",
  line 133, in save_system_state
  scrub_sources=True)

  File "/tmp/ubuntu-release-upgrader-j2xvu5/DistUpgrade/apt_clone.py",
  line 150, in save_state
  self._write_state_sources_list(tar, scrub_sources)

  File "/tmp/ubuntu-release-upgrader-j2xvu5/DistUpgrade/apt_clone.py",
  line 231, in _write_state_sources_list
  "./etc/apt/sources.list")

  File "/tmp/ubuntu-release-upgrader-j2xvu5/DistUpgrade/apt_clone.py",
  line 251, in _add_file_to_tar_with_password_check
  for line in f.readlines():

  File "/usr/lib/python2.7/encodings/ascii.py", line 26, in decode
  return codecs.ascii_decode(input, self.errors)[0]

  UnicodeDecodeError: 'ascii' codec can't decode byte 0xc5 in position
  2604: ordinal not in range(128)

  ProblemType: Bug
  DistroRelease: Ubuntu 13.10
  Package: ubuntu-release-upgrader-core 1:0.205.5
  ProcVersionSignature: Ubuntu 3.11.0-19.33~ppa1-generic-tuxonice 3.11.10.5
  Uname: Linux 3.11.0-19-generic-tuxonice x86_64
  NonfreeKernelModules: fglrx
  ApportVersion: 2.12.5-0ubuntu2.2
  Architecture: amd64
  CrashDB: ubuntu
  Date: Fri Apr 18 11:54:23 2014
  DistributionChannelDescriptor:
   # This is a distribution channel descriptor
   # For more information see http://wiki.ubuntu.com/DistributionChannelDescriptor
   canonical-oem-somerville-oneiric-amd64-20111116-1
  EcryptfsInUse: Yes
  InstallationDate: Installed on 2012-05-07 (711 days ago)
  InstallationMedia: Ubuntu 11.10 "Oneiric" - Build amd64 LIVE Binary 20111116-18:24
  MarkForUpload: True
  PackageArchitecture: all
  SourcePackage: ubuntu-release-upgrader
  Symptom: release-upgrade
  UpgradeStatus: Upgraded to saucy on 2014-04-18 (0 days ago)

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/apt-clone/+bug/1309447/+subscriptions