[Bug 1647101] Re: UnicodeDecodeError in fast-export when author name contains non-ascii characters

[Ratchanan Srirattanamet]

Please SRU this to zesty.

-- 
You received this bug notification because you are a member of Ubuntu
Sponsors Team, which is subscribed to the bug report.
https://bugs.launchpad.net/bugs/1647101

Title:
  UnicodeDecodeError in fast-export when author name contains non-ascii
  characters

Status in python-fastimport:
  Fix Committed
Status in python-fastimport package in Ubuntu:
  Fix Released

Bug description:
  I was able to decipher that it failed when it got to the part of
  history where the commit author's name was "Raúl Núñez". (git-bzr is
  http://github.com/termie/git-bzr-ng)

  $ git bzr clone lp:sakura
  You have not informed bzr of your Launchpad ID, and you must do this to
  write to Launchpad or access private data.  See "bzr help launchpad-login".
  Branched 562 revisions.tching revisions:Finishing stream:Estimate 6041/7652
  14:42:10 Calculating the revisions to include ...
  14:42:10 Starting export of 743 revisions ...
  bzr: ERROR: exceptions.UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 2: ordinal not in range(128)

  Traceback (most recent call last):
    File "/usr/lib/python2.7/site-packages/bzrlib/commands.py", line 930, in exception_to_return_code
      return the_callable(*args, **kwargs)
    File "/usr/lib/python2.7/site-packages/bzrlib/commands.py", line 1121, in run_bzr
      ret = run(*run_argv)
    File "/usr/lib/python2.7/site-packages/bzrlib/commands.py", line 673, in run_argv_aliases
      return self.run(**all_cmd_args)
    File "/usr/lib/python2.7/site-packages/bzrlib/commands.py", line 697, in run
      return self._operation.run_simple(*args, **kwargs)
    File "/usr/lib/python2.7/site-packages/bzrlib/cleanup.py", line 136, in run_simple
      self.cleanups, self.func, *args, **kwargs)
    File "/usr/lib/python2.7/site-packages/bzrlib/cleanup.py", line 166, in _do_with_cleanups
      result = func(*args, **kwargs)
    File "/usr/lib/python2.7/site-packages/bzrlib/plugins/fastimport/cmds.py", line 720, in run
      return exporter.run()
    File "/usr/lib/python2.7/site-packages/bzrlib/plugins/fastimport/exporter.py", line 240, in run
      self.emit_commit(revid, self.ref)
    File "/usr/lib/python2.7/site-packages/bzrlib/plugins/fastimport/exporter.py", line 358, in emit_commit
      self.print_cmd(self._get_commit_command(ref, mark, revobj, file_cmds))
    File "/usr/lib/python2.7/site-packages/bzrlib/plugins/fastimport/exporter.py", line 287, in print_cmd
      self.outf.write("%r\n" % cmd)
    File "/usr/lib/python2.7/site-packages/fastimport/commands.py", line 74, in __repr__
      return self.__bytes__()
    File "/usr/lib/python2.7/site-packages/fastimport/commands.py", line 186, in __bytes__
      return self.to_string(include_file_contents=True)
    File "/usr/lib/python2.7/site-packages/fastimport/commands.py", line 201, in to_string
      author_section = b'\nauthor ' + format_who_when(self.author)
    File "/usr/lib/python2.7/site-packages/fastimport/commands.py", line 504, in format_who_when
      name = utf8_bytes_string(name)
    File "/usr/lib/python2.7/site-packages/fastimport/helpers.py", line 104, in utf8_bytes_string
      return s.encode('utf8')
  UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 2: ordinal not in range(128)

  bzr 2.7.0 on python 2.7.12 (Linux-4.8.8-2-ARCH-x86_64-with-glibc2.2.5)
  arguments: ['/usr/sbin/bzr', 'fast-export', '--plain', '--export-
      marks=/home/forivall/code/repos/sakura/.git/bzr/map/master-bzr', '--git-
      branch=bzr/master',
      '/home/forivall/code/repos/sakura/.git/bzr/repo/master']
  plugins: bash_completion[2.7.0], changelog_merge[2.7.0],
      fastimport[0.14.0dev], grep[2.7.0], launchpad[2.7.0],
      netrc_credential_store[2.7.0], news_merge[2.7.0], po_merge[2.7.0],
      weave_fmt[2.7.0]
  encoding: 'utf-8', fsenc: 'UTF-8', lang: 'en_CA.UTF-8'

  *** Bazaar has encountered an internal error.  This probably indicates a
      bug in Bazaar.  You can help us fix it by filing a bug report at
          https://bugs.launchpad.net/bzr/+filebug
      including this traceback and a description of the problem.
  ERROR:root:bzr export failed

  I fixed it by changing `s.encode('utf8')` to `return unicode(s,
  'utf8').encode('utf8')`, but that should probably be limited to only
  when `sys.getdefaultencoding()` returns 'ascii'

To manage notifications about this bug go to:
https://bugs.launchpad.net/python-fastimport/+bug/1647101/+subscriptions