[MERGE] Test for bug #272444 (symlinks to Unicode file names)

[Andrew Bennetts]

Daniel Clemente wrote:
> Andrew Bennetts <andrew.bennetts at canonical.com> writes:
> 
> >> === modified file 'bzrlib/tests/branch_implementations/test_sprout.py'
> > [...]
> >> +    def test_sprout_with_unicode_symlink(self):
> >> +        # this tests bug #272444
> >> +        # Also tested by TestSetParents.test_unicode_symlink at test_parents.py
> > ..........
> 
> >  * it seems unfortunate that it's possible to commit a revision that
> >    can't then be sprouted.  If we're going to fail, we probably should
> >    fail sooner rather than later.  That's not your fault of course, so
> >    you don't have to fix it in this patch, but it is worth noting.
> 
>   The revision may be correct and the problem may be in the sprout.
>   But do you mean, that maybe WorkingTree.commit() or Commit.commit() should do the right decoding? Maybe that's the place to investigate.

Well, regardless of the reason, if as a user I can do “bzr commit” but then
I cannot “bzr branch” from that because of a unicode error, even on the same
filesystem... then that's a pretty nasty pitfall for users.

I don't know what needs fixing, but I do think that it would be better for
commit to fail than to make an unbranchable/uncheckoutable branch.

Interestingly, when I try from the command line to commit a symlink to adiós I
get a traceback, so there's something different about the test does vs. what
“bzr commit” does.  (And the traceback varies depending on Python version I run
bzr with, and whether or not I override $HOME to be an empty directory!)

Oh, and weirdly, your tests only pass for me on Python 2.5.  They fail with both
2.4 and 2.6.

> >  * it's odd to have a test that claims to duplicate the testing done by
> >    another test.  Is it really a duplicate?  If so, what's the value in
> >    having it?  It seems to be a workingtree issue rather than a branch
> >    issue, so a branch_implementations test does seem to be the wrong
> >    place.
> >
> 
>   I did originally write the test with self.run_bzr("branch ...") and found that it failed with WorkingTreeFormat4, 3 and 2. When I rewrote it with set_parent_trees, I found that only WorkingTreeFormat4 failed. So Aaron Bentley suggested here [1] to test also sprout() since that is what „branch“ uses.
> 
> [1]: http://thread.gmane.org/gmane.comp.version-control.bazaar-ng.general/47300/focus=47848

That seems a bit strange to me, surely it's still the workingtree that's
failing?  When I look at the exceptions behind the KnownFailures in your
branch_implementations test I see that the errors all come from inside
initialize in workingtree_4.py, which seems to confirm my intuition.

Aaron, can you elaborate on why a branch_implementation test for a workingtree
problem makes sense to you?

>   Other than:
> - breaking the long line
> - telling other test's path in a comment
> - explaining in the comment why are there two tests
>   ...should I change something else? I'll send the patch when we decide if we need both tests or not.

Unfortunately, yes: make the tests pass with Python 2.4, and ideally Python 2.6.
And ideally fix the UnicodeWarnings that the tests emit in 2.5.

The branch tests are failing in 2.6 with errors like:

Traceback (most recent call last):
  File "/home/andrew/warthogs/bzr/tmp/bzrlib/tests/branch_implementations/test_sprout.py", line 116, in test_sprout_with_unicode_symlink
    revision = tree.commit('added a link to a Unicode target')
  File "/home/andrew/warthogs/bzr/tmp/bzrlib/decorators.py", line 192, in write_locked
    result = unbound(self, *args, **kwargs)
  File "/home/andrew/warthogs/bzr/tmp/bzrlib/workingtree_4.py", line 237, in commit
    result = WorkingTree3.commit(self, message, revprops, *args, **kwargs)
  File "/home/andrew/warthogs/bzr/tmp/bzrlib/decorators.py", line 192, in write_locked
    result = unbound(self, *args, **kwargs)
  File "/home/andrew/warthogs/bzr/tmp/bzrlib/mutabletree.py", line 197, in commit
    revprops=revprops, *args, **kwargs)
  File "/home/andrew/warthogs/bzr/tmp/bzrlib/commit.py", line 412, in commit
    self._basis_delta)
  File "/home/andrew/warthogs/bzr/tmp/bzrlib/workingtree_4.py", line 1277, in update_basis_by_delta
    self.current_dirstate().update_basis_by_delta(delta, new_revid)
  File "/home/andrew/warthogs/bzr/tmp/bzrlib/dirstate.py", line 1339, in update_basis_by_delta
    inv_to_entry(inv_entry), True))
  File "/home/andrew/warthogs/bzr/tmp/bzrlib/dirstate.py", line 1847, in _inv_entry_to_details
    fingerprint = str(inv_entry.symlink_target or '')
UnicodeEncodeError: 'ascii' codec can't encode character u'\u03a9' in position 0: ordinal not in range(128)

The workingtree test for WorkingTreeFormat4 is failing in the same way in 2.6.
They workingtree tests also fail in 2.4, but in a slightly different way (with a
UnicodeDecodeError slightly earlier instead).

Under 2.5 the tests pass but emit UnicodeWarnings, which probably should be
cleaned up too.  I suspect there's a similar root cause for all these failures
and warnings.

-Andrew.