[PATCH] encoding fix

[Olaf Conradi]

Hi

On 04/04/06, Jan Hudec <bulb at ucw.cz> wrote:
> On Tue, Apr 04, 2006 at 08:43:42 +0200, Olaf Conradi wrote:
> > On 04/04/06, Robert Collins <robertc at robertcollins.net> wrote:
> > > We have multiple uses of getfilesystemencoding in bzrlib. We should
> > > consolidate them so we dont have shotgun surgery with bugs like this.
> >
> > There is only one place in bzrlib where getfilesystemencoding is
> > called. Other two places are in test cases (test_selftest.py and
> > test_msgeditor.py).

The places in test_selftest.py and test_msgeditor.py are harmless as
they would just print None onscreen. So that leaves osutils.py with a
flaw.


> > > Also, a test case is -strongly- desired - encoding issues are some of
> > > the most pernacious bugs we encounter.
> >
> > I would not know how to make a test case for such a lowlevel system
> > function. It's not like you can set it. Either your system returns
> > None or a proper encoding. I think it gets triggered on openbsd
> > systems.
>
> Sure you can:
>
> import sys
> def dummy_getfilesystemencoding():
>     return 'whatever_you_want'
> sys.getfilesystemencoding = dummy_getfilesystemencoding
>
> This is a dynamic language!
>
> (in the test-suite it's a bit more tricky, because you have to actually
> trigger the call to it and you must reinstate the original version, but this
> is the idea)
>
> The other question is, whether the function could ever return an invalid
> encoding (the low-level locale stuff will silently return completely bogus
> data, so it depends on how well python sanitizes them)


Ok, just had another look at it. Redefining could be done, restoring
is also trivial:

save = getattr(sys, "getfilesystemencoding")
sys.getfilesystemencoding = dummy_getfilesystemencoding
osutils.abspath('hello')
sys.getfilesystemencoding = save

But the redefinition of abspath happens at import. Even defining the
above before the import does not seem to work as the import of sys
inside osutils.py creates a new local scope.

After import I could redefine: osutils._fs_enc =
sys.getfilesystemencoding() or 'ascii'

Then the testcase would work, as it would point to my dummy
definition, but then I'd be testing the definition inside the
testcase, which happens to be the same at this point in time. Kind of
defeats a unit test.

Anyone have any other ideas?

We could start a new instance of python in a subshell, setting
nl_langinfo(CODESET) is what determines the encoding at python
startup.

 -Olaf