[Bug 1733553] [NEW] python3-babel requires UTF-8 locale

[Jarda Benkovsky]

Public bug reported:

babel throws an exception when compiling a PO file with non-ascii
characters unless locale is set to some with UTF-8 encoding.

Example when compiling django-filter language catalogs:

$ echo $LC_ALL
C
$ python3 setup.py compile_catalog --directory django_filters/locale/ --locale es_AR --domain django
running compile_catalog
Traceback (most recent call last):
  File "setup.py", line 60, in <module>
    zip_safe=False,
  File "/usr/lib/python3.5/distutils/core.py", line 148, in setup
    dist.run_commands()
  File "/usr/lib/python3.5/distutils/dist.py", line 955, in run_commands
    self.run_command(cmd)
  File "/usr/lib/python3.5/distutils/dist.py", line 974, in run_command
    cmd_obj.run()
  File "/usr/lib/python3/dist-packages/babel/messages/frontend.py", line 134, in run
    catalog = read_po(infile, locale)
  File "/usr/lib/python3/dist-packages/babel/messages/pofile.py", line 205, in read_po
    for lineno, line in enumerate(fileobj.readlines()):
  File "/usr/lib/python3.5/encodings/ascii.py", line 26, in decode
    return codecs.ascii_decode(input, self.errors)[0]
UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 681: ordinal not in range(128)

With UTF-8 locale:

$ LC_ALL=C.UTF-8 python3 setup.py compile_catalog --directory django_filters/locale/ --locale es_AR --domain django
running compile_catalog
compiling catalog 'django_filters/locale/es_AR/LC_MESSAGES/django.po' to 'django_filters/locale/es_AR/LC_MESSAGES/django.mo'

The PO file has an encoding defined, so babel should not strictly needed
it specified in LC_ALL, I think:

...
"Content-Type: text/plain; charset=UTF-8\n"
"Content-Transfer-Encoding: 8bit\n"
...
#: filters.py:51
msgid "This is an exclusion filter"
msgstr "Este es un filtro de exclusión"


(The bug does not manfest with python-babel package)

python3-babel: 1.3+dfsg.1-6
Ubuntu 16.04.2 LTS (in a pbuilder-cowbuilder VM)

** Affects: python-babel (Ubuntu)
     Importance: Undecided
         Status: New

-- 
You received this bug notification because you are a member of Ubuntu
OpenStack, which is subscribed to python-babel in Ubuntu.
https://bugs.launchpad.net/bugs/1733553

Title:
  python3-babel requires UTF-8 locale

Status in python-babel package in Ubuntu:
  New

Bug description:
  babel throws an exception when compiling a PO file with non-ascii
  characters unless locale is set to some with UTF-8 encoding.

  Example when compiling django-filter language catalogs:

  $ echo $LC_ALL
  C
  $ python3 setup.py compile_catalog --directory django_filters/locale/ --locale es_AR --domain django
  running compile_catalog
  Traceback (most recent call last):
    File "setup.py", line 60, in <module>
      zip_safe=False,
    File "/usr/lib/python3.5/distutils/core.py", line 148, in setup
      dist.run_commands()
    File "/usr/lib/python3.5/distutils/dist.py", line 955, in run_commands
      self.run_command(cmd)
    File "/usr/lib/python3.5/distutils/dist.py", line 974, in run_command
      cmd_obj.run()
    File "/usr/lib/python3/dist-packages/babel/messages/frontend.py", line 134, in run
      catalog = read_po(infile, locale)
    File "/usr/lib/python3/dist-packages/babel/messages/pofile.py", line 205, in read_po
      for lineno, line in enumerate(fileobj.readlines()):
    File "/usr/lib/python3.5/encodings/ascii.py", line 26, in decode
      return codecs.ascii_decode(input, self.errors)[0]
  UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 681: ordinal not in range(128)

  With UTF-8 locale:

  $ LC_ALL=C.UTF-8 python3 setup.py compile_catalog --directory django_filters/locale/ --locale es_AR --domain django
  running compile_catalog
  compiling catalog 'django_filters/locale/es_AR/LC_MESSAGES/django.po' to 'django_filters/locale/es_AR/LC_MESSAGES/django.mo'

  The PO file has an encoding defined, so babel should not strictly
  needed it specified in LC_ALL, I think:

  ...
  "Content-Type: text/plain; charset=UTF-8\n"
  "Content-Transfer-Encoding: 8bit\n"
  ...
  #: filters.py:51
  msgid "This is an exclusion filter"
  msgstr "Este es un filtro de exclusión"

  
  (The bug does not manfest with python-babel package)

  python3-babel: 1.3+dfsg.1-6
  Ubuntu 16.04.2 LTS (in a pbuilder-cowbuilder VM)

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/python-babel/+bug/1733553/+subscriptions