[RFC] Summary on test suite failures on OSX

John Arbash Meinel john at arbash-meinel.com
Fri Sep 5 00:03:51 BST 2008


-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Vincent Ladeuil wrote:
> Below are the tests that are erroring or failing on OSX as of
> today. I also include the expected failures specific to OSX.
> 
> ,----
> | ERROR: blackbox.test_export.TestExport.test_tar_export
> |     'NoneType' object has no attribute 'startswith'
> `----
> 
> Not totally diagnosed yet. But the traceback ends up in
> python2.5/tarfile.py and there are a *lot* of differences between
> the stock OSX 10.4 python-2.5.1 and python-2.5.2 on hardy.

certainly sounds like a tarfile discrepency. But without the traceback...


> ,----
> | ERROR: blackbox.test_init.TestInit.test_init_unicode
> |     'ascii' codec can't encode character u'\xb5' in position 18: ordinal not in range(128)
> `----
> 
> Running with LC_CTYPE=en_US.UTF-8 and LC_ALL=en_US.UTF-8 makes it
> pass, what should we do here ?
> 
> That seems to be linked to osutils.get_user_encoding() try to
> work around some bug on darwin:
> 
>     if sys.platform == 'darwin':
>         # work around egregious python 2.4 bug
>         sys.platform = 'posix'
>         try:
>             import locale
>         finally:
>             sys.platform = 'darwin'
>     else:
>         import locale
> 
> I'd love some info about that egregious bug :-/

Specifically, without doing the workaround,
locale.getpreferredencoding() always returns 'MacRoman'. Which isn't
true anymore. Without doing the workaround, setting LANG has no effect
on getpreferredencoding().

The issue is that the filesystem is UTF-8. I'm guessing we are having a
problem printing something out to the user, etc. Which would be a
terminal encoding issue. I know a bug there is that terminal encoding
*is* utf-8 (at least for the stock kernel) but without setting
LANG=....UTF-8 it thinks it is actually MacRoman or somesuch...

So MacPython doesn't really detect its encodings correctly, so we have
to sort of hack around them. (Such as asking users to set LANG.)

I suppose we could just force the user encoding, terminal encoding, etc
on Mac.




> ,----
> | ERROR: tree_implementations.test_test_trees.TestTreeShapes.test_tree_with_merged_utf8(WorkingTreeFormat4)
> |     Working tree "/private/tmp/testbzr-vJk5hc.tmp/bzrlib.tests.tree_implementations.test_test_trees.TestTreeShapes.test_tree_with_merged_utf8(WorkingTreeFormat4)/work/" has uncommitted changes.
...

> 
> Yeah, right, go find the bug in a ~20.000 lines patch 8-(

I'll point you in the right location. It is using a NFC utf-8 string.
Change the test to use a string that can't be combining.

For example instead of using u'\xe5' == å == u'a\u030a' use something
like "\N{omega}".

> 
> And since the test does:
> 
>         self.build_tree([u'tree2/b\xe5r/z\xf7z'])
>         tree2.add([u'b\xe5r/z\xf7z'], [u'z\xf7z-id'.encode('utf-8')])
>         tree2.commit(u'to m\xe9rge', rev_id=u'r\xe9v-2'.encode('utf8'))
> 
> It should be a normalization problem.
> 
> I've yet to understand if the problem is in how the test is
> written of if dirstate lacks some normalization.
> 
> Comments welcome while the investigations continue...
> 

I would change the test to not use a character than plays poorly with
normalization. We have that problem on Mac, but we don't need to test it
at every turn. When we *fix* that, we will write tests explicitly for it.


> ,----
> | FAIL: workingtree_implementations.test_rename_one.TestRenameOne.test_rename_to_denormalised_fails(WorkingTreeFormat4)
> |     (<class 'bzrlib.errors.InvalidNormalization'>, <type 'exceptions.UnicodeEncodeError'>) not raised
> | 
> | FAIL: workingtree_implementations.test_rename_one.TestRenameOne.test_rename_to_denormalised_fails(WorkingTreeFormat3)
> |     (<class 'bzrlib.errors.InvalidNormalization'>, <type 'exceptions.UnicodeEncodeError'>) not raised
> | 
> | FAIL: workingtree_implementations.test_rename_one.TestRenameOne.test_rename_to_denormalised_fails(WorkingTreeFormat2)
> |     (<class 'bzrlib.errors.InvalidNormalization'>, <type 'exceptions.UnicodeEncodeError'>) not raised
> | 
> `----
> 
> These ones fail since they have been introduced (bzr at 2843),
> Robert, can you explain the intent of this test ? Or better its
> consequences.
> 
> I can add a:
>   if osutils.normalizes_filenames():
>      from bzrlib import tests
>      raise tests.TestNotApplicable('OSX normalizes filenames')
> 
> But is this correct ? If it's to guard against invalid filenames
> on the file system, AFAIK, OSX provides that.
> 

Specifically, we want to refuse to add or rename a non-normalized
filename. Eventually we want to relax that restriction. But until we can
get there, we need to filter at add and rename time.

John
=:->


> ,----
> | ERROR: workingtree_implementations.test_workingtree.TestIllegalPaths.test_bad_fs_path(WorkingTreeFormat4)
> |     invalid mode: wb
> | 
> | ERROR: workingtree_implementations.test_workingtree.TestIllegalPaths.test_bad_fs_path(WorkingTreeFormat3)
> |     invalid mode: wb
> | 
> | ERROR: workingtree_implementations.test_workingtree.TestIllegalPaths.test_bad_fs_path(WorkingTreeFormat2)
> |     invalid mode: wb
> | 
> `----
> 
> The same should apply here, I think :)
> 
> Thanks in advance,
> 
>    Vincent
> 
> 

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (Cygwin)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iEYEARECAAYFAkjAaVcACgkQJdeBCYSNAAN/rQCcDEx127mRBLFHCyDrs2QtgKTt
ba4AnA6apPGejP557Yp63Msn9x3I9SNi
=appB
-----END PGP SIGNATURE-----



More information about the bazaar mailing list