[merge] Handle bad LANG

[Martin Pool]

On 24 Aug 2006, John Arbash Meinel <john at arbash-meinel.com> wrote:
> Right now if you set LANG to something unrecognized by python, you get
> an ugly traceback.
> 
> The attached patch changes bzrlib so it falls back to just using 'ascii'
> after emitting a semi-helpful warning.
> 
> I'm not 100% on the english of the warning, but I think it is reasonable.
> 
> This is probably small enough for 0.10, but I'll let Robert decide.

I'd vote for it for 0.10.

I'd tend to call it 'locale' rather than 'LANG', both in the code and
message.  That it's controlled by $LANG is a bit of an implementation
detail.  (And I think on some other unixes it may not be $LANG?)  In
particular on Windows is it $LANG that controls this?  If not the
message may be confusing.  (Or perhaps this error can't occur.)

Even on Linux there are a bunch of LC_* variables which could influence
this.

> +class TestLANG(TestCaseWithTransport):

So, TestLocale, etc.

> +    def test_log_BOGUS(self):
> +        os.environ['LANG'] = 'BOGUS'
> +        out, err = self.run_bzr_subprocess('--no-aliases', '--no-plugins',
> +                                           '-q', 'log', '--log-format=long',
> +                                           'tree')
> +        # XXX: This depends on the exact formatting of a locale.Error
> +        # as the first part of the string. It may be a little tempermental
> +        self.assertEqualDiff("""\
> +WARNING: unsupported locale setting
> +  Could not determine your preferred encoding.
> +  Usually, this is because python does not support your LANG ('BOGUS')
> +  Using 'ascii' encoding.
> +""", err)
> +        self.assertEqualDiff("""\
> +------------------------------------------------------------
> +revno: 1
> +committer: ???? Meinel <juju at info.com>
> +branch nick: tree
> +timestamp: Thu 2006-08-24 20:28:17 +0000
> +message:
> +  Unicode ? commit
> +""", out)

Looks good.

> === modified file NEWS
> --- NEWS
> +++ NEWS
> @@ -23,6 +23,9 @@
>  
>    BUG FIXES:
>  
> +    * Handle when LANG is not recognized by python. Emit a warning, but
> +      just revert to using 'ascii'. (John Arbash Meinel, 35392)
> +
>      * Active FTP transport now works as intended. (ghozzy, #56472)
>  
>      * Really fix mutter() so that it won't ever raise a UnicodeError.
> 

Just add a '#35392' so that we can grep for them.

> === modified file bzrlib/__init__.py
> --- bzrlib/__init__.py
> +++ bzrlib/__init__.py
> @@ -21,16 +21,34 @@
>  
>  import os
>  import sys
> -if sys.platform == 'darwin':
> -    # work around egregious python 2.4 bug
> -    sys.platform = 'posix'
> -    import locale
> -    sys.platform = 'darwin'
> -else:
> -    import locale
> -# XXX: This probably belongs in osutils instead
> -user_encoding = locale.getpreferredencoding() or 'ascii'
> -del locale
> +
> +
> +def get_user_encoding():
> +    if sys.platform == 'darwin':
> +        # work around egregious python 2.4 bug
> +        sys.platform = 'posix'
> +        import locale
> +        sys.platform = 'darwin'
> +    else:
> +        import locale
> +    # XXX: This probably belongs in osutils instead
> +    user_encoding = None
> +    try:
> +        user_encoding = locale.getpreferredencoding()
> +    except locale.Error, e:
> +        sys.stderr.write('WARNING: %s\n'
> +                         '  Could not determine your preferred encoding.\n'
> +                         '  Usually, this is because python does not support'
> +                         ' your LANG (%r)\n'
> +                         "  Using 'ascii' encoding.\n"
> +                         % (e, os.environ.get('LANG')))
> +
> +    if user_encoding is None:
> +        return 'ascii'
> +    return user_encoding
> +
> +user_encoding = get_user_encoding()
> +

Since we're changing this, should we perhaps put it in osutils or
bzrlib.locale, to keep the init file small?  (We would presumably need a
deprecated property forwarding to the new one.)

If the intention is that the method be executed only once to initialize
it, it should have an underscore.  Otherwise people will get the warning
repeatedly.

I'd suggest this message text:

  bzr: warning: %s
  Could not determine what text encoding to use.
  This error usually means your Python interpreter doesn't support
  the locale set by $LANG (%s).
  Continuing with ascii encoding.

Thanks

-- 
Martin