bzr, unicode in file names and Mac OS X

[Hinnerk Haardt]

Hi,

I encountered three possibly related tracebacks while trying to to  
move a version history from svn/svk to bazaar using tailor. They have  
in common that the file names contain umlauts and are created on Mac  
OS X.

The first two can be reproduced only on Mac OS X, the third on Linux  
using the attached .tgz:


1) System: Mac OS X 10.4.9, LANG=de_DE.UTF-8, LC_CTYPE=de_DE.UTF-8,  
shell is bash in Terminal.app:

    Description: "bzr init", add directory with umlaut using finder,  
"bzr add", traceback:

> $ bzr init test
> $ cd test
> $ open .
> [use the finder to create a directory named "ü"]
> $ bzr add .
> added "ü"
> bzr: ERROR: exceptions.KeyError: u'u\u0308'
>
> Traceback (most recent call last):
>   File
> "/opt/local/Library/Frameworks/Python.framework/Versions/2.4/lib/ 
> python2.4/site-packages/bzrlib/commands.py",
> line 650, in run_bzr_catch_errors
>     return run_bzr(argv)
>   File
> "/opt/local/Library/Frameworks/Python.framework/Versions/2.4/lib/ 
> python2.4/site-packages/bzrlib/commands.py",
> line 612, in run_bzr
>     ret = run(*run_argv)
>   File
> "/opt/local/Library/Frameworks/Python.framework/Versions/2.4/lib/ 
> python2.4/site-packages/bzrlib/commands.py",
> line 304, in run_argv_aliases
>     return self.run(**all_cmd_args)
>   File
> "/opt/local/Library/Frameworks/Python.framework/Versions/2.4/lib/ 
> python2.4/site-packages/bzrlib/builtins.py",
> line 372, in run
>     action=action, save=not dry_run)
>   File
> "/opt/local/Library/Frameworks/Python.framework/Versions/2.4/lib/ 
> python2.4/site-packages/bzrlib/add.py",
> line 148, in smart_add
>     return smart_add_tree(tree, file_list, recurse,
> action=action, save=save)
>   File
> "/opt/local/Library/Frameworks/Python.framework/Versions/2.4/lib/ 
> python2.4/site-packages/bzrlib/add.py",
> line 175, in smart_add_tree
>     action=action, save=save)
>   File
> "/opt/local/Library/Frameworks/Python.framework/Versions/2.4/lib/ 
> python2.4/site-packages/bzrlib/add.py",
> line 305, in _smart_add_tree
>     this_ie = parent_ie.children[directory.base_path]
> KeyError: u'u\u0308'
>
> bzr 0.15.0 on python 2.4.3.final.0 (darwin)
> arguments: ['/opt/local/bin/bzr', 'add', '.']
>
> ** please send this report to bazaar at lists.ubuntu.com

 From ~/.bzr.log:

> this is a debug log for diagnosing/reporting problems in bzr
> you can delete or truncate this file, or include sections in
> bug reports to bazaar at lists.canonical.com
>
> bzr arguments: [u'add', u'.']
> looking for plugins in /Users/user/.bazaar/plugins
> looking for plugins in /opt/local/Library/Frameworks/ 
> Python.framework/Versions/2.4/lib/python2.4/site-packages/bzrlib/ 
> plugins
> Plugin name __init__ already loaded
> Plugin name __init__ already loaded
> encoding stdout as sys.stdout encoding 'UTF-8'
> got branch format Bazaar-NG branch format 5
> opening working tree '/Volumes/Backup/test/test'
> smart add of [u'.'], originally [u'.']
> skip control directory '.bzr'
> Traceback (most recent call last):
>   File "/opt/local/Library/Frameworks/Python.framework/Versions/2.4/ 
> lib/python2.4/site-packages/bzrlib/commands.py", line 650, in  
> run_bzr_catch_errors
>     return run_bzr(argv)
>   File "/opt/local/Library/Frameworks/Python.framework/Versions/2.4/ 
> lib/python2.4/site-packages/bzrlib/commands.py", line 612, in run_bzr
>     ret = run(*run_argv)
>   File "/opt/local/Library/Frameworks/Python.framework/Versions/2.4/ 
> lib/python2.4/site-packages/bzrlib/commands.py", line 304, in  
> run_argv_aliases
>     return self.run(**all_cmd_args)
>   File "/opt/local/Library/Frameworks/Python.framework/Versions/2.4/ 
> lib/python2.4/site-packages/bzrlib/builtins.py", line 372, in run
>     action=action, save=not dry_run)
>   File "/opt/local/Library/Frameworks/Python.framework/Versions/2.4/ 
> lib/python2.4/site-packages/bzrlib/add.py", line 148, in smart_add
>     return smart_add_tree(tree, file_list, recurse, action=action,  
> save=save)
>   File "/opt/local/Library/Frameworks/Python.framework/Versions/2.4/ 
> lib/python2.4/site-packages/bzrlib/add.py", line 175, in  
> smart_add_tree
>     action=action, save=save)
>   File "/opt/local/Library/Frameworks/Python.framework/Versions/2.4/ 
> lib/python2.4/site-packages/bzrlib/add.py", line 305, in  
> _smart_add_tree
>     this_ie = parent_ie.children[directory.base_path]
> KeyError: u'u\u0308'
>
> return code 3




2) System: Mac OS X 10.4.9, LANG=de_DE.UTF-8, LC_CTYPE=de_DE.UTF-8,  
shell is bash in Terminal.app:

    Description: "bzr init", add file with umlaut, "bzr add", "bzr  
ci", add another file with umlaut, "bzr add .", traceback:

> $ bzr init test
> $ cd test
> $ cat /etc/fstab.hd > ü-ü.txt
> $ bzr add .
> added "ü-ü.txt"
> $ bzr ci -m "Test 1"
> added ü-ü.txt
> Committed revision 1.
> $ cat /etc/fstab.hd > ü.txt
> $ bzr add .
> added "ü-ü.txt"
> bzr: ERROR:Traceback (most recent call last):
>   File "/opt/local/bin/bzr", line 95, in ?
>     exit_val = bzrlib.commands.main(sys.argv)
>   File "/opt/local/Library/Frameworks/Python.framework/Versions/2.4/ 
> lib/python2.4/site-packages/bzrlib/commands.py", line 643, in main
>     ret = run_bzr_catch_errors(argv)
>   File "/opt/local/Library/Frameworks/Python.framework/Versions/2.4/ 
> lib/python2.4/site-packages/bzrlib/commands.py", line 656, in  
> run_bzr_catch_errors
>     trace.report_exception(sys.exc_info(), sys.stderr)
>   File "/opt/local/Library/Frameworks/Python.framework/Versions/2.4/ 
> lib/python2.4/site-packages/bzrlib/trace.py", line 286, in  
> report_exception
>     report_user_error(exc_info, err_file)
>   File "/opt/local/Library/Frameworks/Python.framework/Versions/2.4/ 
> lib/python2.4/site-packages/bzrlib/trace.py", line 304, in  
> report_user_error
>     print >>err_file, "bzr: ERROR:", str(exc_info[1])
>   File "/opt/local/Library/Frameworks/Python.framework/Versions/2.4/ 
> lib/python2.4/site-packages/bzrlib/errors.py", line 94, in __str__
>     return str(s)
> UnicodeEncodeError: 'ascii' codec can't encode character u'\xfc' in  
> position 0: ordinal not in range(128)

from ~/.bzr.log:

> bzr arguments: [u'init', u'test']
> looking for plugins in /Users/user/.bazaar/plugins
> looking for plugins in /opt/local/Library/Frameworks/ 
> Python.framework/Versions/2.4/lib/python2.4/site-packages/bzrlib/ 
> plugins
> Plugin name __init__ already loaded
> Plugin name __init__ already loaded
> encoding stdout as sys.stdout encoding 'UTF-8'
> created control directory in file:///Volumes/Backup/test/test/
> creating repository in file:///Volumes/Backup/test/test/.bzr/.
> creating branch <bzrlib.branch.BzrBranchFormat5 object at 0x5ecd90>  
> in file:///Volumes/Backup/test/test/.bzr/
> got branch format Bazaar-NG branch format 5
> got branch format Bazaar-NG branch format 5
> trying to create missing lock '/Volumes/Backup/test/test/.bzr/ 
> checkout/dirstate'
> opening working tree '/Volumes/Backup/test/test'
> return code 0
>
> bzr arguments: [u'add', u'.']
> looking for plugins in /Users/user/.bazaar/plugins
> looking for plugins in /opt/local/Library/Frameworks/ 
> Python.framework/Versions/2.4/lib/python2.4/site-packages/bzrlib/ 
> plugins
> Plugin name __init__ already loaded
> Plugin name __init__ already loaded
> encoding stdout as sys.stdout encoding 'UTF-8'
> got branch format Bazaar-NG branch format 5
> opening working tree '/Volumes/Backup/test/test'
> smart add of [u'.'], originally [u'.']
> skip control directory '.bzr'
> return code 0
>
> bzr arguments: [u'ci', u'-m', u'Test 1']
> looking for plugins in /Users/user/.bazaar/plugins
> looking for plugins in /opt/local/Library/Frameworks/ 
> Python.framework/Versions/2.4/lib/python2.4/site-packages/bzrlib/ 
> plugins
> Plugin name __init__ already loaded
> Plugin name __init__ already loaded
> encoding stdout as sys.stdout encoding 'UTF-8'
> got branch format Bazaar-NG branch format 5
> opening working tree '/Volumes/Backup/test/test'
> preparing to commit
> Selecting files for commit with filter None
> [16517] Thu 18:30:57.491 INFO: added ü-ü.txt
> added revision_id {haardt at mac.com-20070412163057-bwpczij9mc3q3kr0}
> [16517] Thu 18:30:58.246 INFO: Committed revision 1.
> return code 0
>
> bzr arguments: [u'add', u'.']
> looking for plugins in /Users/user/.bazaar/plugins
> looking for plugins in /opt/local/Library/Frameworks/ 
> Python.framework/Versions/2.4/lib/python2.4/site-packages/bzrlib/ 
> plugins
> Plugin name __init__ already loaded
> Plugin name __init__ already loaded
> encoding stdout as sys.stdout encoding 'UTF-8'
> got branch format Bazaar-NG branch format 5
> opening working tree '/Volumes/Backup/test/test'
> smart add of [u'.'], originally [u'.']
> skip control directory '.bzr'
> Traceback (most recent call last):
>   File "/opt/local/Library/Frameworks/Python.framework/Versions/2.4/ 
> lib/python2.4/site-packages/bzrlib/commands.py", line 650, in  
> run_bzr_catch_errors
>     return run_bzr(argv)
>   File "/opt/local/Library/Frameworks/Python.framework/Versions/2.4/ 
> lib/python2.4/site-packages/bzrlib/commands.py", line 612, in run_bzr
>     ret = run(*run_argv)
>   File "/opt/local/Library/Frameworks/Python.framework/Versions/2.4/ 
> lib/python2.4/site-packages/bzrlib/commands.py", line 304, in  
> run_argv_aliases
>     return self.run(**all_cmd_args)
>   File "/opt/local/Library/Frameworks/Python.framework/Versions/2.4/ 
> lib/python2.4/site-packages/bzrlib/builtins.py", line 372, in run
>     action=action, save=not dry_run)
>   File "/opt/local/Library/Frameworks/Python.framework/Versions/2.4/ 
> lib/python2.4/site-packages/bzrlib/add.py", line 148, in smart_add
>     return smart_add_tree(tree, file_list, recurse, action=action,  
> save=save)
>   File "/opt/local/Library/Frameworks/Python.framework/Versions/2.4/ 
> lib/python2.4/site-packages/bzrlib/add.py", line 175, in  
> smart_add_tree
>     action=action, save=save)
>   File "/opt/local/Library/Frameworks/Python.framework/Versions/2.4/ 
> lib/python2.4/site-packages/bzrlib/add.py", line 299, in  
> _smart_add_tree
>     __add_one(tree, inv, parent_ie, directory, kind, action)
>   File "/opt/local/Library/Frameworks/Python.framework/Versions/2.4/ 
> lib/python2.4/site-packages/bzrlib/add.py", line 390, in __add_one
>     inv.add(entry)
>   File "/opt/local/Library/Frameworks/Python.framework/Versions/2.4/ 
> lib/python2.4/site-packages/bzrlib/inventory.py", line 1157, in add
>     entry.name))
> BzrError: <unprintable instance object>




3) System: Linux, Kernel 2.6.20, LANG=de_DE.UTF-8,  
LC_CTYPE=de_DE.UTF-8, shell is bash:

    Description: "bzr init", scp or extract .tgz from Mac OS X, "bzr  
add", traceback:

> $ bzr init test
> $ cd test/
> $ scp -r user at mac:/test .
> 100%    0     0.0KB/s   00:00
> $ ls -l test/
> drwxr-xr-x 2 user group 4096 12. Apr 18:47 ä
> -rw-r--r-- 1 user group    0 12. Apr 18:47 ü.txt
> $ bzr add .
> added test
> added "test/ä"
> bzr: ERROR: Path u'a\u0308' is not unicode normalized

This works with the files from a .tgz too, you find an archive  
(Umlaut.tgz) attached to this mail.

from ~/.bzr.log:

> this is a debug log for diagnosing/reporting problems in bzr
> you can delete or truncate this file, or include sections in
> bug reports to bazaar at lists.canonical.com
>
> bzr arguments: [u'init', u'test']
> looking for plugins in /home/user/.bazaar/plugins
> looking for plugins in /usr/lib/python2.4/site-packages/bzrlib/plugins
> Plugin name __init__ already loaded
> Plugin name __init__ already loaded
> encoding stdout as sys.stdout encoding 'UTF-8'
> created control directory in file:///tmp/test/
> creating repository in file:///tmp/test/.bzr/.
> creating branch <bzrlib.branch.BzrBranchFormat5 object at  
> 0xb7b438ac> in file:///tmp/test/.bzr/
> got branch format Bazaar-NG branch format 5
> got branch format Bazaar-NG branch format 5
> trying to create missing lock '/tmp/test/.bzr/checkout/dirstate'
> opening working tree '/tmp/test'
> return code 0
>
> bzr arguments: [u'add', u'.']
> looking for plugins in /home/user/.bazaar/plugins
> looking for plugins in /usr/lib/python2.4/site-packages/bzrlib/plugins
> Plugin name __init__ already loaded
> Plugin name __init__ already loaded
> encoding stdout as sys.stdout encoding 'UTF-8'
> got branch format Bazaar-NG branch format 5
> opening working tree '/tmp/test'
> smart add of [u'.'], originally [u'.']
> skip control directory '.bzr'
> Traceback (most recent call last):
>   File "/usr/lib/python2.4/site-packages/bzrlib/commands.py", line  
> 650, in run_bzr_catch_errors
>     return run_bzr(argv)
>   File "/usr/lib/python2.4/site-packages/bzrlib/commands.py", line  
> 612, in run_bzr
>     ret = run(*run_argv)
>   File "/usr/lib/python2.4/site-packages/bzrlib/commands.py", line  
> 304, in run_argv_aliases
>     return self.run(**all_cmd_args)
>   File "/usr/lib/python2.4/site-packages/bzrlib/builtins.py", line  
> 372, in run
>     action=action, save=not dry_run)
>   File "/usr/lib/python2.4/site-packages/bzrlib/add.py", line 148,  
> in smart_add
>     return smart_add_tree(tree, file_list, recurse, action=action,  
> save=save)
>   File "/usr/lib/python2.4/site-packages/bzrlib/add.py", line 175,  
> in smart_add_tree
>     action=action, save=save)
>   File "/usr/lib/python2.4/site-packages/bzrlib/add.py", line 299,  
> in _smart_add_tree
>     __add_one(tree, inv, parent_ie, directory, kind, action)
>   File "/usr/lib/python2.4/site-packages/bzrlib/add.py", line 389,  
> in __add_one
>     file_id=file_id)
>   File "/usr/lib/python2.4/site-packages/bzrlib/inventory.py", line  
> 1402, in make_entry
>     raise errors.InvalidNormalization(name)
> InvalidNormalization: Path u'a\u0308' is not unicode normalized
>
> return code 3

-------------- next part --------------
A non-text attachment was scrubbed...
Name: Umlaut.tgz
Type: application/octet-stream
Size: 173 bytes
Desc: not available
Url : https://lists.ubuntu.com/archives/bazaar/attachments/20070412/37b75292/attachment-0001.obj