[Bug 1647101] [NEW] UnicodeDecodeError in fast-export when author name contains non-ascii characters

[Launchpad Bug Tracker]

You have been subscribed to a public bug by Unit 193 (unit193):

I was able to decipher that it failed when it got to the part of history
where the commit author's name was "Raúl Núñez". (git-bzr is
http://github.com/termie/git-bzr-ng)

$ git bzr clone lp:sakura
You have not informed bzr of your Launchpad ID, and you must do this to
write to Launchpad or access private data.  See "bzr help launchpad-login".
Branched 562 revisions.tching revisions:Finishing stream:Estimate 6041/7652
14:42:10 Calculating the revisions to include ...
14:42:10 Starting export of 743 revisions ...
bzr: ERROR: exceptions.UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 2: ordinal not in range(128)

Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/bzrlib/commands.py", line 930, in exception_to_return_code
    return the_callable(*args, **kwargs)
  File "/usr/lib/python2.7/site-packages/bzrlib/commands.py", line 1121, in run_bzr
    ret = run(*run_argv)
  File "/usr/lib/python2.7/site-packages/bzrlib/commands.py", line 673, in run_argv_aliases
    return self.run(**all_cmd_args)
  File "/usr/lib/python2.7/site-packages/bzrlib/commands.py", line 697, in run
    return self._operation.run_simple(*args, **kwargs)
  File "/usr/lib/python2.7/site-packages/bzrlib/cleanup.py", line 136, in run_simple
    self.cleanups, self.func, *args, **kwargs)
  File "/usr/lib/python2.7/site-packages/bzrlib/cleanup.py", line 166, in _do_with_cleanups
    result = func(*args, **kwargs)
  File "/usr/lib/python2.7/site-packages/bzrlib/plugins/fastimport/cmds.py", line 720, in run
    return exporter.run()
  File "/usr/lib/python2.7/site-packages/bzrlib/plugins/fastimport/exporter.py", line 240, in run
    self.emit_commit(revid, self.ref)
  File "/usr/lib/python2.7/site-packages/bzrlib/plugins/fastimport/exporter.py", line 358, in emit_commit
    self.print_cmd(self._get_commit_command(ref, mark, revobj, file_cmds))
  File "/usr/lib/python2.7/site-packages/bzrlib/plugins/fastimport/exporter.py", line 287, in print_cmd
    self.outf.write("%r\n" % cmd)
  File "/usr/lib/python2.7/site-packages/fastimport/commands.py", line 74, in __repr__
    return self.__bytes__()
  File "/usr/lib/python2.7/site-packages/fastimport/commands.py", line 186, in __bytes__
    return self.to_string(include_file_contents=True)
  File "/usr/lib/python2.7/site-packages/fastimport/commands.py", line 201, in to_string
    author_section = b'\nauthor ' + format_who_when(self.author)
  File "/usr/lib/python2.7/site-packages/fastimport/commands.py", line 504, in format_who_when
    name = utf8_bytes_string(name)
  File "/usr/lib/python2.7/site-packages/fastimport/helpers.py", line 104, in utf8_bytes_string
    return s.encode('utf8')
UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 2: ordinal not in range(128)

bzr 2.7.0 on python 2.7.12 (Linux-4.8.8-2-ARCH-x86_64-with-glibc2.2.5)
arguments: ['/usr/sbin/bzr', 'fast-export', '--plain', '--export-
    marks=/home/forivall/code/repos/sakura/.git/bzr/map/master-bzr', '--git-
    branch=bzr/master',
    '/home/forivall/code/repos/sakura/.git/bzr/repo/master']
plugins: bash_completion[2.7.0], changelog_merge[2.7.0],
    fastimport[0.14.0dev], grep[2.7.0], launchpad[2.7.0],
    netrc_credential_store[2.7.0], news_merge[2.7.0], po_merge[2.7.0],
    weave_fmt[2.7.0]
encoding: 'utf-8', fsenc: 'UTF-8', lang: 'en_CA.UTF-8'

*** Bazaar has encountered an internal error.  This probably indicates a
    bug in Bazaar.  You can help us fix it by filing a bug report at
        https://bugs.launchpad.net/bzr/+filebug
    including this traceback and a description of the problem.
ERROR:root:bzr export failed

I fixed it by changing `s.encode('utf8')` to `return unicode(s,
'utf8').encode('utf8')`, but that should probably be limited to only
when `sys.getdefaultencoding()` returns 'ascii'

** Affects: python-fastimport
     Importance: Medium
     Assignee: Jelmer Vernooij (jelmer)
         Status: Fix Committed

** Affects: python-fastimport (Ubuntu)
     Importance: Undecided
         Status: Confirmed

-- 
UnicodeDecodeError in fast-export when author name contains non-ascii characters
https://bugs.launchpad.net/bugs/1647101
You received this bug notification because you are a member of Ubuntu Sponsors Team, which is subscribed to the bug report.