Using bzr with pycurl on Windows

vila v.ladeuil+lp at free.fr
Thu May 26 09:49:41 UTC 2011


>>>>> Eli Zaretskii <eliz at gnu.org> writes:

<snip/>

    > Done, see below.  It looks like the problem is "404 Not Found", is
    > that right?

I don't think so, they all seem ~normal.

<snip/>

    > There are no such credentials in the log.  I hope I'm not wasting
    > everyone's time by making some stupid cockpit error.  However, I
    > did succeed with "git clone" from the same address, both on this
    > machine and on another one, so it doesn't seem to be a simple
    > mis-configuration problem.

Thanks for the additional info.

    > Thanks.  Here's the full log from .bzr.log:

    >   Thu 2011-05-26 10:21:29 +0300
    >   0.329  bazaar version: 2.3.3
    >   0.329  bzr arguments: [u'-Dhttp', u'branch', u'https://github.com/dlitz/pycrypto.git']
    >   0.439  looking for plugins in C:/Documents and Settings/p0009057/Application Data/bazaar/2.0/plugins
    >   0.439  looking for plugins in C:/Program Files/Bazaar/plugins
    >   1.206  encoding stdout as sys.stdout encoding 'cp862'
    >   1.253  failed to import pycurl: No module named pycurl
    >   1.253  failed to instantiate transport <bzrlib.registry._LazyObjectGetter object at 106bb20, module='bzrlib.transport.http._pycurl' attribute='PyCurlTransport' imported=False> for 'https://github.com/dlitz/pycrypto.git': DependencyNotPresent(Unable to import library "pycurl": No module named pycurl)
    >   1.786  bzr-svn: using Subversion 1.6.6 (), subvertpy 0.7.5
    >   2.350  * About to connect() to github.com:443

First connection.

    >   4.684  > OPTIONS /dlitz/pycrypto.git/

Not a bzr request, I know bzr-svn starts with such a request, no idea
about bzr-git.

One trick to try would be: 'BZR_DISABLE_PLUGINS=svn bzr ...' hmm, or
whatever the equivalent syntax is on windows (i.e. disable the bzr-svn
plugin just for this command and compare the logs).

    >   4.684  > Host: github.com
    >> Accept: */*
    >> User-Agent: bzr/2.3.3 (urllib)
    >> Connection: Keep-Alive
    >> Pragma: no-cache
    >> Cache-Control: max-age=0

    >   5.154  < HTTP/1.1 404 Not Found
    >   5.154  < Server: nginx/0.7.67
    >   < Date: Thu, 26 May 2011 07:20:02 GMT
    >   < Content-Type: text/html; charset=utf-8
    >   < Connection: keep-alive
    >   < Status: 404 Not Found
    >   < X-Runtime: 15ms
    >   < Content-Length: 19315
    >   < Set-Cookie: csrf_id=<OBFUSCATED>; path=/
    >   < Set-Cookie: _gh_sess=<OBFUSCATED>; path=/; expires=Fri, 01 Jan 2021 00:00:00 GMT; secure; HttpOnly
    >   < Cache-Control: no-cache

    >   5.185  > POST /dlitz/pycrypto.git/.bzr/smart

That's the usual bzr probe for a smart server.

    >   5.185  > Content-Length: 6
    >> Connection: Keep-Alive
    >> Accept: */*
    >> User-Agent: bzr/2.3.3 (urllib)
    >> Host: github.com
    >> Pragma: no-cache
    >> Cache-Control: max-age=0
    >> Content-Type: application/octet-stream

    >   5.874  < HTTP/1.1 404 Not Found
    >   5.874  < Server: nginx/0.7.67
    >   < Date: Thu, 26 May 2011 07:20:03 GMT
    >   < Content-Type: text/html; charset=utf-8
    >   < Connection: keep-alive
    >   < Status: 404 Not Found
    >   < Content-Length: 19367
    >   < Set-Cookie: csrf_id=<OBFUSCATED>; path=/
    >   < Set-Cookie: _gh_sess=<OBFUSCATED>; path=/; expires=Fri, 01 Jan 2021 00:00:00 GMT; secure; HttpOnly
    >   < Cache-Control: no-cache

    >   5.890  > GET /dlitz/pycrypto.git/.bzr/branch-format
    >   5.890  > Host: github.com
    >> Accept: */*
    >> User-Agent: bzr/2.3.3 (urllib)
    >> Connection: Keep-Alive
    >> Pragma: no-cache
    >> Cache-Control: max-age=0

    >   6.219  < HTTP/1.1 404 Not Found
    >   6.219  < Server: nginx/0.7.67
    >   < Date: Thu, 26 May 2011 07:20:03 GMT
    >   < Content-Type: text/html; charset=utf-8
    >   < Connection: keep-alive
    >   < Status: 404 Not Found
    >   < Content-Length: 19362
    >   < Set-Cookie: csrf_id=<OBFUSCATED>; path=/
    >   < Set-Cookie: _gh_sess=<OBFUSCATED>; path=/; expires=Fri, 01 Jan 2021 00:00:00 GMT; secure; HttpOnly
    >   < Cache-Control: no-cache

    >   6.219  > HEAD /dlitz/pycrypto.git/info/refs

This looks like bzr-git is probing now.

    >   6.219  > Host: github.com
    >> Accept: */*
    >> User-Agent: bzr/2.3.3 (urllib)
    >> Connection: Keep-Alive
    >> Pragma: no-cache
    >> Cache-Control: max-age=0

    >   81.563  * About to connect() to github.com:443

Huh ? A new connection, which implies something bad happened to the
previous one :-/

That's probably where the bug is, we're supposed to clean the pipe
before reconnecting and it seems we don't ?


    >   82.174  > HEAD /dlitz/pycrypto.git/info/refs
    >   82.174  > Host: github.com
    >> Accept: */*
    >> User-Agent: bzr/2.3.3 (urllib)
    >> Connection: Keep-Alive
    >> Pragma: no-cache
    >> Cache-Control: max-age=0

    >   157.533  Transferred: 61kB (0.4kB/s r:60kB w:1kB)
    >   157.533  Traceback (most recent call last):
    >     File "bzrlib\commands.pyo", line 926, in exception_to_return_code
    >     File "bzrlib\commands.pyo", line 1126, in run_bzr
    >     File "bzrlib\commands.pyo", line 691, in run_argv_aliases
    >     File "bzrlib\commands.pyo", line 713, in run
    >     File "bzrlib\cleanup.pyo", line 135, in run_simple
    >     File "bzrlib\cleanup.pyo", line 165, in _do_with_cleanups
    >     File "bzrlib\builtins.pyo", line 1183, in run
    >     File "bzrlib\bzrdir.pyo", line 806, in open_tree_or_branch
    >     File "bzrlib\bzrdir.pyo", line 717, in open
    >     File "bzrlib\bzrdir.pyo", line 747, in open_from_transport
    >     File "bzrlib\lazy_import.pyo", line 128, in __call__
    >     File "bzrlib\transport\__init__.pyo", line 1670, in do_catching_redirections
    >     File "bzrlib\bzrdir.pyo", line 734, in find_format
    >     File "bzrlib\controldir.pyo", line 754, in find_format
    >     File "C:/Program Files/Bazaar/plugins\git\__init__.py", line 130, in probe_transport

Right, that's indeed bzr-git probing. 

    >     File "bzrlib\transport\__init__.pyo", line 569, in has_any
    >     File "bzrlib\transport\http\_urllib.pyo", line 164, in has
    >     File "bzrlib\transport\http\_urllib.pyo", line 157, in _head
    >     File "bzrlib\transport\http\_urllib.pyo", line 79, in _perform
    >     File "urllib2.pyo", line 391, in open
    >     File "urllib2.pyo", line 409, in _open
    >     File "urllib2.pyo", line 369, in _call_chain
    >     File "bzrlib\transport\http\_urllib2_wrappers.pyo", line 763, in https_open
    >     File "bzrlib\transport\http\_urllib2_wrappers.pyo", line 669, in do_open
    >     File "bzrlib\transport\http\_urllib2_wrappers.pyo", line 584, in retry_or_raise

Darn, the debug info is only available by modifying the DEBUG variable
at the beginning of the module. That sucks, blame me :-/

There are comments there saying:

                    # httplib.BadStatusLine and
                    # httplib.UnknownProtocol indicates that a
                    # bogus server was encountered or a bad
                    # connection (i.e. transient errors) is
                    # experimented, we have already retried once
                    # for that request so we raise the exception.

But in your case that's not transient right ?

So we're back to square one :-/ We need to see what the server is
returning there.

Trying to reproduce locally with bzr-git trunk tip, I get: 

InvalidHttpResponse: Invalid http response for
https://github.com/dlitz/pycrypto.git/info/refs: Unable to handle http
code 0: 

and activating the debug locally finally reveals:

  reply: 'incomplete headers'

so we get 'incomplete headers' where we expect a proper HTTP status
line... that's not a nice server behavior there :-/ Note that this is
coming straight from the python httplib module which means the string is
sent as-is on the wire instead of a proper:

  reply: 'HTTP/1.1 404 Not Found\r\n'

or whatever HTTP way to report errors.

It doesn't give a clue about which header is incomplete either.

While there I've filed http://pad.lv/788530 so we get a bit more debug
info in such cases:

$ bzr branch -Dhttp https://github.com/dlitz/pycrypto.git
bzr: ERROR: Invalid http response for https://github.com/dlitz/pycrypto.git/info/refs: Bad status line received: BadStatusLine('incomplete headers',)

I vaguely suspect something related to the cookies but that really:
1) is a wild guess,
2) doesn't explain why the server stops speaking HTTP lingo...

           Vincent





More information about the bazaar mailing list