[MERGE][1.14][355273] Fix OSX test failures

[Jean-Francois Roy]

On Apr 7, 2009, at 02:15, Vincent Ladeuil wrote:

>    jfroy> tty issues aside, special care must indeed be taken on Mac  
> OS X
>    jfroy> w/r to  unicode decomposition when sending and receiving  
> file
>    jfroy> paths.
>
> That's exactly where we have problems (some may be due to python
> itself) and John Meinel has summarized that to the list at
> numerous occasions.
>
> That's the part I don't intend to fix right now but instead try
> to get back to the point we were excluding these issues as a
> *first* step. But even that may well be outside of this bug
> scope.
>
>    jfroy> The encoding will always be UTF-8 for all filesystems
>    jfroy> at the API level however (it's a POSIX compliance
>    jfroy> requirement and an implementation
>    jfroy> detail).
>
> I seem to remember that part of the problem was indeed that OSX
> didn't use the same decomposition (NKD or NKFD ? instead of NKC
> NKFC ?) as other systems which led to specific problems (but John
> knows better, see the archive for details).

The issue is with HFS+ specifically, and it requires NFD  
decomposition. It's not a bad idea in and of itself as it allows  
faster searches while ignoring diacritics (e.g. search for "etagere"  
will still find "étagère"). Details are at http://developer.apple.com/technotes/tn/tn1150.html#UnicodeSubtleti 
es.

>
>    jfroy> Decomposition is something Subversion does wrong and
>    jfroy> it has bitten be in the past when accented characters.
>
> And AFAIR we don't intend to do better (yet) as that will require
> special handling for OSX anyway (again, based on John's remarks,
> I seem to recall that OSX use a normalization that is more
> expensive than the others and we didn't want to make all OSes
> incur that cost).

I would expect Bazaar to do this correctly. Normalization will only  
come into play when doing naïve byte-wise comparison of Unicode  
strings, or when hashing Unicode strings (be they str or unicode  
objects).

Regardless, it's a cost the client will have to pay on Mac OS X, since  
Bazaar should probably standardize on the more common NFC form.

>
>    jfroy> In addition, the one other quirk I am aware of are "Icon\r"
>    jfroy> files,  which have an empty data fork but some data in the
>    jfroy> resource fork. Most  tools can't deal with the Mac line  
> break
>    jfroy> (CR) in the file name,  including bzr's .bzrignore  
> mechanism. I'm
>    jfroy> pretty sure Bazaar doesn't  support multiple file forks or
>    jfroy> extended attributes either, but that's  a different matter.
>
> Geee, I thought that OSX was getting rid of the fork madness for
> good, Apple is pretty good at making my old macs unsupported (I
> still have a G3 and a G4 running there), it may be time for them
> to obsolete the fork mechanism given that all Apple applications
> seems to do well without it (AFAICS).

Icon\r are basically irrelevant to Bazaar. They're used to store  
custom icons for folders IIRC. Ideally, I would expand the .bzrignore  
file to access escape sequences, but realistically I would perhaps go  
as far as hard-ignoring Icon\r.

> And where is that "Icon\r" used ? And how does the Finder
> displays it ? Can I issue 'ls -al Icon\r' ?

You can paste an image in the Get Info view on a folder. Then, if you  
ls -la that folder, you should get an entry like this:

-rw-r--r--@  1 me  admin       0 Apr  7 10:14 Icon?

UNIX tools don't deal with the CR character very well either...

>
>    Vincent