[Bug 1243065] Re: Invalid encoding returned for ca at valencia locale

[Brian Murray]

** Changed in: ubuntu-release-upgrader (Ubuntu)
       Status: New => Triaged

** Changed in: ubuntu-release-upgrader (Ubuntu)
   Importance: Undecided => High

-- 
You received this bug notification because you are a member of Ubuntu
Foundations Bugs, which is subscribed to ubuntu-release-upgrader in
Ubuntu.
https://bugs.launchpad.net/bugs/1243065

Title:
  Invalid encoding returned for ca at valencia locale

Status in Python:
  Confirmed
Status in “ubuntu-release-upgrader” package in Ubuntu:
  Triaged

Bug description:
  While upgrading from 13.04 to 13.10 using locale ca at valencia, the
  process fails with the following traceback:

  Traceback (most recent call last):

  File "/tmp/ubuntu-release-upgrader-vh1jgw/saucy", line 10, in 
  <module> 
  sys.exit(main()) 

  File 
  "/tmp/ubuntu-release-upgrader-vh1jgw/DistUpgrade/DistUpgradeMain.py", 
  line 243, in main 
  if app.run(): 

  File 
  "/tmp/ubuntu-release-upgrader-vh1jgw/DistUpgrade/DistUpgradeController.py", 
  line 1791, in run 
  return self.fullUpgrade() 

  File 
  "/tmp/ubuntu-release-upgrader-vh1jgw/DistUpgrade/DistUpgradeController.py", 
  line 1744, in fullUpgrade 
  if not self.askDistUpgrade(): 

  File 
  "/tmp/ubuntu-release-upgrader-vh1jgw/DistUpgrade/DistUpgradeController.py", 
  line 996, in askDistUpgrade 
  self.cache.required_download) 

  File 
  "/tmp/ubuntu-release-upgrader-vh1jgw/DistUpgrade/DistUpgradeViewText.py", 
  line 165, in confirmChanges 
  downloadSize, actions) 

  File 
  "/tmp/ubuntu-release-upgrader-vh1jgw/DistUpgrade/DistUpgradeView.py", 
  line 387, in confirmChanges 
  downloadSizeStr = downloadSizeStr.decode(ENCODING) 

  LookupError: unknown encoding: utf_8_valencia

  Looking at it in more detail, this comes from the following code in
  DistUpgrade/DistUpgradeView.py [1]

  try:
      locale.setlocale(locale.LC_ALL, "")
      (code, ENCODING) = locale.getdefaultlocale()
  except:
      logging.exception("getting the encoding failed")
      ENCODING = "utf-8"   #pyflakes

  if not ENCODING:
      ENCODING = "utf-8"

  And digging deeper, it seems getdefaultlocale does not return the
  correct encoding when the locale is set to ca_ES

  >>> import locale
  >>> locale.setlocale(locale.LC_ALL, "")
  'LC_CTYPE=ca_ES.UTF-8 at valencia;LC_NUMERIC=ca_ES.UTF-8;LC_TIME=ca_ES.UTF-8;LC_COLLATE=ca_ES.UTF-8 at valencia;LC_MONETARY=ca_ES.UTF-8;LC_MESSAGES=ca_ES.UTF-8 at valencia;LC_PAPER=ca_ES.UTF-8;LC_NAME=ca_ES.UTF-8;LC_ADDRESS=ca_ES.UTF-8;LC_TELEPHONE=ca_ES.UTF-8;LC_MEASUREMENT=ca_ES.UTF-8;LC_IDENTIFICATION=ca_ES.UTF-8'
  >>> locale.getdefaultlocale()
  ('ca_ES', 'utf_8_valencia')

  Instead of 'utf_8_valencia', here I'd expect 'UTF-8' to be returned.
  The returned locale is also wrong: it should be ca_ES at valencia.

  While this needs to be fixed upstream, it might be worth adding a
  workaround (and a check for a valid encoding) in ubuntu-release-
  upgrader in the meantime.

  This will also affect other @ locales, such as sr at Latin (Serbian with
  Latin encoding).

  [1] http://bazaar.launchpad.net/~ubuntu-core-dev/ubuntu-release-
  upgrader/trunk/view/head:/DistUpgrade/DistUpgradeView.py#L41

To manage notifications about this bug go to:
https://bugs.launchpad.net/python/+bug/1243065/+subscriptions