[MERGE] Fix bug #229076 by fixing header names before sending the request.

[Vincent Ladeuil]

>>>>> "john" == John Arbash Meinel <john at arbash-meinel.com> writes:

    john> Vincent Ladeuil wrote:
    >> Hi,
    >> 
    >> Never leave a FIXME without giving detailed explanations... or
    >> you'll end up doing all the research work again :)
    >> 
    >> See patch for explanations, roughly,
    >> python2.4/2.5/httplib/urllib/urllib2 internally capitalized
    >> (User-agent) header names (or lowercase (user-agent) them, it
    >> depends) when in fact at least one server/proxy on the earth
    >> expect them to be titled (User-Agent) and abort connections
    >> otherwise. Unfortunately, it is used by www.gnome.org* ;)
    >> 
    >> It turns out python2.5 urllib2 in some recent version (didn't
    >> search which one exactly) fixed it by using title.
    >> 
    >> Unfortunately it did it in a function I had to override to handle
    >> persistent connections (among other things).
    >> 
    >> Anyway, thanks for reading until here, I'm just being a bit
    >> verbose to respect my introduction ;-)
    >> 
    >> Vincent
    >> 
    >> *: The bug is fixed, but I'm curious as a cat, so I'd like to
    >> know what is being used there, any pointers greatly appreciated.
    >> 
    >> 
    john> === modified file 'bzrlib/transport/http/_urllib2_wrappers.py'
    john> --- bzrlib/transport/http/_urllib2_wrappers.py	2008-04-24 07:22:53 +0000
    john> +++ bzrlib/transport/http/_urllib2_wrappers.py	2008-05-17 17:53:47 +0000
    john> @@ -30,6 +30,10 @@

    john> ~ And a custom Request class that lets us track redirections, and
    john> ~ handle authentication schemes.
    john> +
    john> +For coherency with python libraries, we use capitalized header names throughout
    john> +the code, even if the header names will be titled just before sending the
    john> +request (see AbstractHTTPHandler.do_open).
    john> ~ """

    john> ~ DEBUG = 0
    john> @@ -407,10 +411,6 @@
    john> ~     _default_headers = {'Pragma': 'no-cache',
    john> ~                         'Cache-control': 'max-age=0',
    john> ~                         'Connection': 'Keep-Alive',
    john> -                        # FIXME: Spell it User-*A*gent once we
    john> -                        # know how to properly avoid bogus
    john> -                        # urllib2 using capitalize() for headers
    john> -                        # instead of title(sp?).
    john> ~                         'User-agent': 'bzr/%s (urllib)' % bzrlib_version,
    john> ~                         'Accept': '*/*',
    john> ~                         }

    john> ^- You added the comment, but I see "Cache-control" rather than "Cache-Control"
    john> and still have "User-Agent".

See above "For coherency with python libraries, we use
capitalized header names".

If I use 'User-Agent' (titled) here, the request will contain
'User-Agent: bzr-blah-blah' and 'User-agent: python-blah-blah'
(capitalized) because the internals checks are done with
capitalized or lower cased headers names.

Yes, this is messy.

No I don't intend to get rid of both urllib2 and
_urllib2_wrappers and provide an httplib2 implementation to fix
this bug ;-)

But yes, the more work-arounds I implement the more I'm tempted
to do so...

    john> Otherwise

    john> BB:approve

Thanks.

    john> I'm also curious why gnome is being that strict,

Me too, martin said in the bug report it will contact mneptok, I
hope he can tell us.

I think this is a bug on their side that is worth fixing. But
without knowing what software is causing it, it's hard to report
such a bug.

    john> I don't remember the spec being that strict.

It isn't. As Robert pointed out the spec says they are
case-insensitive.

        Vincent