Access control

Wed Feb 28 22:54:43 GMT 2007

>>>>> "Jeff" == Jeffrey Cunningham <jcunningham at medea.sea.boeing.com> writes:

    Jeff> Vincent Ladeuil wrote:
    >> 
    >> Did I understand correctly ?
    >> 
    >> You are able to configure your apache server but you need to go
    >> through the proxy to access it ?
    >> 
    >> Are you able to configure the proxy too ?

    Jeff> I can configure my apache server, but the proxy server stands
    Jeff> between this LAN and the internet and is outside my control. The
    Jeff> issues I was having earlier with the proxy server were with
    Jeff> trying to access sites outside the LAN which have to go through
    Jeff> the proxy server. You are correct: I have no need of going
    Jeff> through the proxy server to access my own apache server.

    Jeff> It sounds like bzr is using the proxy server whether I need it or not.

    >> 
    >> Can you try to disable the proxy for that server :
    >> 
    >> no_proxy=medea.sea.boeing.com
    >> 
    >> And by the way, what are your proxy settings for mozilla ?

    Jeff> Mozilla is set up for manual proxy configuration, same proxy,
    Jeff> with localhost and 127.0.0.1 exempted.

Ok.

    Jeff> Here is what I get with no_proxy as above:

Let's look at that.

    Jeff> jcunningham at medea ~/junk $ no_proxy=medea.sea.boeing.com bzr
    Jeff> branch
    Jeff> http+urllib://jeff:password@medea.sea.boeing.com/~jcunningham/docs
    Jeff> Will unbind ftp_open for 'http://www-blv-proxy.boeing.com:31060'
    Jeff> Will unbind http_open for 'http://www-blv-proxy.boeing.com:31060'
    Jeff> Will unbind no_open for 'medea.sea.boeing.com'
    Jeff> Will bind http_request for 'http://www-blv-proxy.boeing.com:31060'
    Jeff> connect: (medea.sea.boeing.com, 80)

Direct connection to the right host.

    Jeff> send: 'GET /~jcunningham/docs/.bzr/branch-format
    Jeff> HTTP/1.1\r\nAccept-Encoding: identity\r\nConnection:
    Jeff> Keep-Alive\r\nAccept: */*\r\nUser-agent: bzr/0.14.0
    Jeff> (urllib)\r\nHost: medea.sea.boeing.com\r\nPragma:
    Jeff> no-cache\r\nCache-control: max-age=0\r\n\r\n'
    Jeff> Request sent: [<bzrlib.transport.http._urllib2_wrappers.Request
    Jeff> instance at 0xb7c3994c>]
    Jeff> reply: 'HTTP/1.1 401 Authorization Required\r\n'
    Jeff> header: Date: Wed, 28 Feb 2007 21:09:13 GMT
    Jeff> header: Server: Apache
    Jeff> header: WWW-Authenticate: Basic realm="sarat"

realm... hmmm... that *should* fallback internally to no realm
but that may also be the cause of the problem.

    Jeff> Note: if I drop this url into Mozilla's url box it authenticates
    Jeff> properly

Showing you the 'sarat' realm in the dialog box I presume ?

<snip/>

    >> So here we go through the proxy and we reached
    >> medea.sea.boeing.com, are you sure you see nothing in the apache
    >> logs for that ?

    Jeff> There is nothing in either the access_log or the error_log except
    Jeff> the Mozilla browser lines. Actually, there's nothing in the
    Jeff> error_log at all past this morning when I stopped the favicon.ico
    Jeff> error by touching an empty file.

Ok.

    >> 
    >> I can't determine if the 400 error code for pycurl came from the
    >> proxy or the apache server.
    >> 
    >> Thanks for your patience, tele-debugging such a config
    >> is... interesting ;-)
    >> 
    >> By the way if you could upgrade your bzr it will be easier to
    >> send you patches or if you tell me what exact version you're
    >> using I can base patches on it.
    >> 

    Jeff> I just upgraded it via bzr (and the proxy). Worked great!

    >> I still don't know where the problem is, but pycurl and urllib
    >> behaving differently for basic authorization is really strange.
    >> 
    >> Well, let's try to divide the problem: can you do the same test
    >> locally on your apache server using http://localhost so that we
    >> can rule out the proxy ? If it works with:
    >> - basic auth for urllib,
    >> - basic or digest for pycurl
    >> 
    >> Then we could have a look at the proxy.

    Jeff> Lets see, I need to re-hack the debugging lines and recompile bzr
    Jeff> first...Okay. Here's what I get:

    Jeff> jcunningham at medea ~/junk $ bzr branch
    Jeff> http+urllib://jeff:password@localhost/~jcunningham/d   ocs
    Jeff> Will unbind ftp_open for 'http://www-blv-proxy.boeing.com:31060'
    Jeff> Will unbind http_open for 'http://www-blv-proxy.boeing.com:31060'
    Jeff> Will unbind no_open for 'medea.sea.boeing.com'
    Jeff> Will bind http_request for 'http://www-blv-proxy.boeing.com:31060'
    Jeff> set_proxy http_request for 'http://www-blv-proxy.boeing.com:31060'
    Jeff> set_proxy: proxy set to http://www-blv-proxy.boeing.com:31060
    Jeff> connect: (www-blv-proxy.boeing.com, 31060)

Wrong we go to the proxy.

    Jeff> send: 'GET http://localhost/~jcunningham/docs/.bzr/branch-format
    Jeff> HTTP/1.1\r\nAccept-Encoding: identity\r\nConnection:
    Jeff> Keep-Alive\r\nAccept: */*\r\nUser-agent: bzr/0.15.0dev0
    Jeff> (urllib)\r\nHost: localhost\r\nPragma:
    Jeff> no-cache\r\nCache-control: max-age=0\r\n\r\n' 
    Jeff> Request sent: [<bzrlib.transport.http._urllib2_wrappers.Request
    Jeff> instance at 0xb7c4902c>] 
    Jeff> reply: 'HTTP/1.1 301 Moved Permanently\r\n'
    Jeff> header: Server: BlueCoat-Security-Appliance
    Jeff> header: Location:http://134.52.202.110
    Jeff> header: Connection: Close For status: [301]

Eeerk a redirection now ! You just want to drive me nuts don't you ;-)

And we are redirected directly to the host instead of the file we
want on the host, that's wrong. Deadly wrong.

    Jeff> , will ready body, length: None Receives response:
    Jeff> <bzrlib.transport.http._urllib2_wrappers.Response
    Jeff> instance at 0xb778ee0 c> For:
    Jeff> 'GET'('http://localhost/~jcunningham/docs/.bzr/branch-format')
    Jeff> Create addinfourl: <addinfourl at -1216811572 whose fp
    Jeff> = <socket._fileobject object at 0xb
    7b42aac> 
    Jeff>   For: 'GET'('http://localhost/~jcunningham/docs/.bzr/branch-format')
    Jeff> Redirected to: http://134.52.202.110

The proxy send us a wrong redirection, the Location header should
be the redirected URL not a host.

Is that your localhost address ? Then you should add it to the
no_proxy var because we don't try to resolve host names to IP
addresses >-/

no_proxy=localhost,134.52.202.110

    Jeff> set_proxy http_request for
    Jeff> 'http://www-blv-proxy.boeing.com:31060' set_proxy:
    Jeff> proxy set to http://www-blv-proxy.boeing.com:31060

    Jeff> connect: (www-blv-proxy.boeing.com, 31060) 

To the proxy again, I think bzr is lost here ;-)

    Jeff> send: 'GET http://134.52.202.110
    Jeff> HTTP/1.1\r\nAccept-Encoding: identity\r\nConnection:
    Jeff> Keep -Alive\r\nAccept: */*\r\nUser-agent:
    Jeff> bzr/0.15.0dev0 (urllib)\r\nHost: 134.52.202.110\r\nPr
    Jeff> agma: no-cache\r\nCache-control: max-age=0\r\n\r\n'
    Jeff> Request sent:
    Jeff> [<bzrlib.transport.http._urllib2_wrappers.Request
    Jeff> instance at 0xb778ee6c>] reply: 'HTTP/1.1 200 OK\r\n'
    Jeff> header: Connection: Keep-Alive header: Date: Wed, 28
    Jeff> Feb 2007 21:25:12 GMT header: Server: Apache header:
    Jeff> Last-Modified: Mon, 15 May 2006 16:39:39 GMT header:
    Jeff> ETag: "10fe76-5a3-552108c0" header: Accept-Ranges:
    Jeff> bytes header: Content-Length: 1443 header:
    Jeff> Content-Type: text/html header: Age: 0 Receives
    Jeff> response:
    Jeff> <bzrlib.transport.http._urllib2_wrappers.Response
    Jeff> instance at 0xb77900e c> For:
    Jeff> 'GET'('http://134.52.202.110') Create addinfourl:
    Jeff> <addinfourl at -1216806740 whose fp =
    Jeff> <socket._fileobject object at 0xb
    7819e2c> 
    Jeff>   For: 'GET'('http://134.52.202.110')
    Jeff> bzr: ERROR: Unknown branch format:

You bet ! The Apache home page is surely not a known branch format :-)

<snip/>

    Jeff> Looks like it is still trying to use the proxy, even on
    Jeff> localhost. So, now I'll try it with no_proxy=localhost bzr ....

    Jeff> jcunningham at medea ~/junk $ no_proxy=localhost bzr branch
    Jeff> http+urllib://jeff:password@localhost/~jcunningham/docs
    Jeff> Will unbind ftp_open for 'http://www-blv-proxy.boeing.com:31060'
    Jeff> Will unbind http_open for 'http://www-blv-proxy.boeing.com:31060'
    Jeff> Will unbind no_open for 'localhost'
    Jeff> Will bind http_request for 'http://www-blv-proxy.boeing.com:31060'
    Jeff> connect: (localhost, 80)

Connection to localhost. Correct.

    Jeff> 
    Jeff> send: 'GET /~jcunningham/docs/.bzr/branch-format
    Jeff> HTTP/1.1\r\nAccept-Encoding: identity\r\nConnection:
    Jeff> Keep-Alive\r\nAccept: */*\r\nUser-agent: bzr/0.15.0dev0
    Jeff> (urllib)\r\nHost: localhost\r\nPragma:
    Jeff> no-cache\r\nCache-control: max-age=0\r\n\r\n' Request
    Jeff> sent: [<bzrlib.transport.http._urllib2_wrappers.Request
    Jeff> instance at 0xb7c3202c>] reply: 'HTTP/1.1 401
    Jeff> Authorization Required\r\n' header: Date: Wed, 28 Feb
    Jeff> 2007 21:26:27 GMT header: Server: Apache header:
    Jeff> WWW-Authenticate: Basic realm="sarat" header:
    Jeff> Content-Length: 460 header: Keep-Alive: timeout=15,
    Jeff> max=100 header: Connection: Keep-Alive header:
    Jeff> Content-Type: text/html; charset=iso-8859-1 For status:
    Jeff> [401], will ready body, length: [460] Consumed body:
    Jeff> [<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
    Jeff> <html><head> <title>401 Authorization Required</title>
    Jeff> </head><body> <h1>Authorization Required</h1> <p>This
    Jeff> server could not verify that you are authorized to
    Jeff> access the document requested.  Either you supplied the
    Jeff> wrong credentials (e.g., bad password), or your browser
    Jeff> doesn't understand how to supply the credentials
    Jeff> required.</p> <hr> <address>Apache Server at localhost
    Jeff> Port 80</address> </body></html>
    Jeff> ]
    Jeff> Receives response:
    Jeff> <bzrlib.transport.http._urllib2_wrappers.Response instance at
    0xb7777dec> 
    Jeff>   For: 'GET'('http://localhost/~jcunningham/docs/.bzr/branch-format')
    Jeff> Create addinfourl: <addinfourl at -1216905812 whose fp =
    Jeff> <socket._fileobject object at 0xb7b2baac>>
    Jeff>   For: 'GET'('http://localhost/~jcunningham/docs/.bzr/branch-format')
    Jeff> bzr: ERROR: Invalid http response for
    Jeff> http://localhost/~jcunningham/docs/.bzr/branch-format: Unable to
    Jeff> handle http code 401: Authorization Required

Light bulb ! I think I get it, it's still a bit unclear to
explain now, I'll try to keep you informed tomorrow, it's related
to the way we store auth info internally for the urllib.

If I'm right pycurl should do better with http://localhost.

There is still something unclear with the proxy, but let's try to
solve the auth bug first.

    Jeff> It still looks like it is trying to go through the proxy. Could
    Jeff> it be ignoring the no_proxy setting on the command line? I'll try
    Jeff> undefining it altogether:

    Jeff> jcunningham at medea ~/junk $ export http_proxy=
    Jeff> jcunningham at medea ~/junk $ env | grep http_proxy

try 

env | grep -i proxy

    Jeff> http_proxy=
    Jeff> jcunningham at medea ~/junk $ bzr branch
    Jeff> http+urllib://jeff:password@127.0.0.1/~jcunningham/docs
    Jeff> Will unbind ftp_open for 'http://www-blv-proxy.boeing.com:31060'

A proxy variable is still lying around....

    Jeff> Will unbind no_open for 'medea.sea.boeing.com' connect:
    Jeff> (127.0.0.1, 80)

but we connect directly to localhost anyway, so maybe a no_proxy
is still set.

<snip/>

    Jeff> Well, you have a good evening in any event.

:-)

        Vincent

P.S.: I'll be on vacations from friday and for a week, I'm not
sure I can find the time to solve that before, but rest assured
that if you can wait my return, I'll find a solution.