Access control

Wed Feb 28 21:32:10 GMT 2007

Vincent Ladeuil wrote:
> 
> Did I understand correctly ?
> 
> You are able to configure your apache server but you need to go
> through the proxy to access it ?
> 
> Are you able to configure the proxy too ?

I can configure my apache server, but the proxy server stands between 
this LAN and the internet and is outside my control. The issues I was 
having earlier with the proxy server were with trying to access sites 
outside the LAN which have to go through the proxy server. You are 
correct: I have no need of going through the proxy server to access my 
own apache server.

It sounds like bzr is using the proxy server whether I need it or not.

> 
> Can you try to disable the proxy for that server :
> 
> no_proxy=medea.sea.boeing.com
> 
> And by the way, what are your proxy settings for mozilla ?

Mozilla is set up for manual proxy configuration, same proxy, with 
localhost and 127.0.0.1 exempted.

Here is what I get with no_proxy as above:

jcunningham at medea ~/junk $ no_proxy=medea.sea.boeing.com bzr branch 
http+urllib://jeff:password@medea.sea.boeing.com/~jcunningham/docs
Will unbind ftp_open for 'http://www-blv-proxy.boeing.com:31060'
Will unbind http_open for 'http://www-blv-proxy.boeing.com:31060'
Will unbind no_open for 'medea.sea.boeing.com'
Will bind http_request for 'http://www-blv-proxy.boeing.com:31060'
connect: (medea.sea.boeing.com, 80)
send: 'GET /~jcunningham/docs/.bzr/branch-format 
HTTP/1.1\r\nAccept-Encoding: identity\r\nConnection: 
Keep-Alive\r\nAccept: */*\r\nUser-agent: bzr/0.14.0 (urllib)\r\nHost: 
medea.sea.boeing.com\r\nPragma: no-cache\r\nCache-control: 
max-age=0\r\n\r\n'
Request sent: [<bzrlib.transport.http._urllib2_wrappers.Request instance 
at 0xb7c3994c>]
reply: 'HTTP/1.1 401 Authorization Required\r\n'
header: Date: Wed, 28 Feb 2007 21:09:13 GMT
header: Server: Apache
header: WWW-Authenticate: Basic realm="sarat"
header: Content-Length: 471
header: Keep-Alive: timeout=15, max=100
header: Connection: Keep-Alive
header: Content-Type: text/html; charset=iso-8859-1
For status: [401], will ready body, length:  [471]
Consumed body: [<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
<html><head>
<title>401 Authorization Required</title>
</head><body>
<h1>Authorization Required</h1>
<p>This server could not verify that you
are authorized to access the document
requested.  Either you supplied the wrong
credentials (e.g., bad password), or your
browser doesn't understand how to supply
the credentials required.</p>
<hr>
<address>Apache Server at medea.sea.boeing.com Port 80</address>
</body></html>
]
Receives response: <bzrlib.transport.http._urllib2_wrappers.Response 
instance at 0xb77bc2ac>
   For: 
'GET'('http://medea.sea.boeing.com/~jcunningham/docs/.bzr/branch-format')
Create addinfourl: <addinfourl at -1216626068 whose fp = 
<socket._fileobject object at 0xb7bacbc4>>
   For: 
'GET'('http://medea.sea.boeing.com/~jcunningham/docs/.bzr/branch-format')
bzr: ERROR: Invalid http response for 
http://medea.sea.boeing.com/~jcunningham/docs/.bzr/branch-format: Unable 
to handle http code 401: Authorization Required

Note: if I drop this url into Mozilla's url box it authenticates 
properly through apache and servers up the directory:

http://jeff:password@medea.sea.boeing.com/~jcunningham/docs

> 
> <snip/>
>     Jeff> browser doesn't understand how to supply
>     Jeff> the credentials required.</p>
>     Jeff> <hr>
>     Jeff> <address>Apache Server at medea.sea.boeing.com Port 80</address>
>     Jeff> </body></html>
>     Jeff> ]
> 
> So here we go through the proxy and we reached
> medea.sea.boeing.com, are you sure you see nothing in the apache
> logs for that ?

There is nothing in either the access_log or the error_log except the 
Mozilla browser lines. Actually, there's nothing in the error_log at all 
past this morning when I stopped the favicon.ico error by touching an 
empty file.

> 
> I can't determine if the 400 error code for pycurl came from the
> proxy or the apache server.
> 
> Thanks for your patience, tele-debugging such a config
> is... interesting ;-)
> 
> By the way if you could upgrade your bzr it will be easier to
> send you patches or if you tell me what exact version you're
> using I can base patches on it.
> 

I just upgraded it via bzr (and the proxy). Worked great!

> I still don't know where the problem is, but pycurl and urllib
> behaving differently for basic authorization is really strange.
> 
> Well, let's try to divide the problem: can you do the same test
> locally on your apache server using http://localhost so that we
> can rule out the proxy ? If it works with:
> - basic auth for urllib,
> - basic or digest for pycurl
> 
> Then we could have a look at the proxy.

Lets see, I need to re-hack the debugging lines and recompile bzr 
first...Okay. Here's what I get:

jcunningham at medea ~/junk $ bzr branch 
http+urllib://jeff:password@localhost/~jcunningham/d 
     ocs
Will unbind ftp_open for 'http://www-blv-proxy.boeing.com:31060'
Will unbind http_open for 'http://www-blv-proxy.boeing.com:31060'
Will unbind no_open for 'medea.sea.boeing.com'
Will bind http_request for 'http://www-blv-proxy.boeing.com:31060'
set_proxy http_request for 'http://www-blv-proxy.boeing.com:31060'
set_proxy: proxy set to http://www-blv-proxy.boeing.com:31060
connect: (www-blv-proxy.boeing.com, 31060)
send: 'GET http://localhost/~jcunningham/docs/.bzr/branch-format 
HTTP/1.1\r\nAccept-Encodi                        ng: 
identity\r\nConnection: Keep-Alive\r\nAccept: */*\r\nUser-agent: 
bzr/0.15.0dev0 (urlli                        b)\r\nHost: 
localhost\r\nPragma: no-cache\r\nCache-control: max-age=0\r\n\r\n'
Request sent: [<bzrlib.transport.http._urllib2_wrappers.Request instance 
at 0xb7c4902c>]
reply: 'HTTP/1.1 301 Moved Permanently\r\n'
header: Server: BlueCoat-Security-Appliance
header: Location:http://134.52.202.110
header: Connection: Close
For status: [301], will ready body, length:  None
Receives response: <bzrlib.transport.http._urllib2_wrappers.Response 
instance at 0xb778ee0                        c>
   For: 'GET'('http://localhost/~jcunningham/docs/.bzr/branch-format')
Create addinfourl: <addinfourl at -1216811572 whose fp = 
<socket._fileobject object at 0xb                        7b42aac>>
   For: 'GET'('http://localhost/~jcunningham/docs/.bzr/branch-format')
Redirected to: http://134.52.202.110
set_proxy http_request for 'http://www-blv-proxy.boeing.com:31060'
set_proxy: proxy set to http://www-blv-proxy.boeing.com:31060
connect: (www-blv-proxy.boeing.com, 31060)
send: 'GET http://134.52.202.110 HTTP/1.1\r\nAccept-Encoding: 
identity\r\nConnection: Keep                        -Alive\r\nAccept: 
*/*\r\nUser-agent: bzr/0.15.0dev0 (urllib)\r\nHost: 134.52.202.110\r\nPr 
                        agma: no-cache\r\nCache-control: max-age=0\r\n\r\n'
Request sent: [<bzrlib.transport.http._urllib2_wrappers.Request instance 
at 0xb778ee6c>]
reply: 'HTTP/1.1 200 OK\r\n'
header: Connection: Keep-Alive
header: Date: Wed, 28 Feb 2007 21:25:12 GMT
header: Server: Apache
header: Last-Modified: Mon, 15 May 2006 16:39:39 GMT
header: ETag: "10fe76-5a3-552108c0"
header: Accept-Ranges: bytes
header: Content-Length: 1443
header: Content-Type: text/html
header: Age: 0
Receives response: <bzrlib.transport.http._urllib2_wrappers.Response 
instance at 0xb77900e                        c>
   For: 'GET'('http://134.52.202.110')
Create addinfourl: <addinfourl at -1216806740 whose fp = 
<socket._fileobject object at 0xb                        7819e2c>>
   For: 'GET'('http://134.52.202.110')
bzr: ERROR: Unknown branch format: '<!DOCTYPE html PUBLIC "-//W3C//DTD 
XHTML 1.1//EN"\n\t\ 
t"http://www.w3.org/TR/xhtml11/DTD/xhtml11.dtd">\n<html 
xmlns="http://www.w3.org/1999/xhtm 
l">\n\t<head>\n\t\t<title>Test Page for Apache 
Installation</title>\n\t</head>\n\t\n\t<bod 
y>\n\t\t<p>If you can see this, it means that the installation of the 
<a\n\t\thref="http:/ 
/www.apache.org/foundation/preFAQ.html">Apache web 
server</a>\n\t\tsoftware on this system                         was 
successful. You may now add content to this\n\t\tdirectory and replace 
this page.</p>                        \n\t\t\n\t\t<hr style="width: 
100%; height: 3px;" />\n\t\t\n\t\t<h2 style="text-align: cen 
             ter">Seeing this instead of the website you 
expected?</h2>\n\t\t\n\t\t<p>This page is here 
because the site administrator has changed the \n\t\tconfiguration of 
this web server. Pl                        ease <strong>contact the 
person\n\t\tresponsible for maintaining this server with question 
                  s.</strong>\n\t\tThe Apache Software Foundation, which 
wrote the web server software\n\t\t                        this site 
administrator is using, has nothing to do with\n\t\tmaintaining this 
site and ca                        nnot help resolve 
configuration\n\t\tissues.</p>\n\t\t\n\t\t<hr style="width: 100%; height 
                        : 3px;" />\n\t\t\n\t\t<p>The Apache 
documentation is available \n\t\t<a href="http://httpd 
       .apache.org/docs-2.0/">online</a> or has been installed\n\t\t<a 
href="/manual/">locally</a 
 >.</p>\n\t\t\n\t\t<p>You are free to use the image below on an 
Apache-powered web\n\t\tser                        ver. Thanks for using 
Apache!</p>\n\t\t\n\t\t<div style="text-align: center"><img src="apa 
                     che_pb.gif" alt="" /></div>\n\t</body>\n</html>\n'

Looks like it is still trying to use the proxy, even on localhost. So, 
now I'll try it with no_proxy=localhost bzr ....

jcunningham at medea ~/junk $ no_proxy=localhost bzr branch 
http+urllib://jeff:password@localhost/~jcunningham/docs
Will unbind ftp_open for 'http://www-blv-proxy.boeing.com:31060'
Will unbind http_open for 'http://www-blv-proxy.boeing.com:31060'
Will unbind no_open for 'localhost'
Will bind http_request for 'http://www-blv-proxy.boeing.com:31060'
connect: (localhost, 80)
send: 'GET /~jcunningham/docs/.bzr/branch-format 
HTTP/1.1\r\nAccept-Encoding: identity\r\nConnection: 
Keep-Alive\r\nAccept: */*\r\nUser-agent: bzr/0.15.0dev0 
(urllib)\r\nHost: localhost\r\nPragma: no-cache\r\nCache-control: 
max-age=0\r\n\r\n'
Request sent: [<bzrlib.transport.http._urllib2_wrappers.Request instance 
at 0xb7c3202c>]
reply: 'HTTP/1.1 401 Authorization Required\r\n'
header: Date: Wed, 28 Feb 2007 21:26:27 GMT
header: Server: Apache
header: WWW-Authenticate: Basic realm="sarat"
header: Content-Length: 460
header: Keep-Alive: timeout=15, max=100
header: Connection: Keep-Alive
header: Content-Type: text/html; charset=iso-8859-1
For status: [401], will ready body, length:  [460]
Consumed body: [<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
<html><head>
<title>401 Authorization Required</title>
</head><body>
<h1>Authorization Required</h1>
<p>This server could not verify that you
are authorized to access the document
requested.  Either you supplied the wrong
credentials (e.g., bad password), or your
browser doesn't understand how to supply
the credentials required.</p>
<hr>
<address>Apache Server at localhost Port 80</address>
</body></html>
]
Receives response: <bzrlib.transport.http._urllib2_wrappers.Response 
instance at 0xb7777dec>
   For: 'GET'('http://localhost/~jcunningham/docs/.bzr/branch-format')
Create addinfourl: <addinfourl at -1216905812 whose fp = 
<socket._fileobject object at 0xb7b2baac>>
   For: 'GET'('http://localhost/~jcunningham/docs/.bzr/branch-format')
bzr: ERROR: Invalid http response for 
http://localhost/~jcunningham/docs/.bzr/branch-format: Unable to handle 
http code 401: Authorization Required

It still looks like it is trying to go through the proxy. Could it be 
ignoring the no_proxy setting on the command line? I'll try undefining 
it altogether:

jcunningham at medea ~/junk $ export http_proxy=
jcunningham at medea ~/junk $ env | grep http_proxy
http_proxy=
jcunningham at medea ~/junk $ bzr branch 
http+urllib://jeff:password@127.0.0.1/~jcunningham/docs
Will unbind ftp_open for 'http://www-blv-proxy.boeing.com:31060'
Will unbind no_open for 'medea.sea.boeing.com'
connect: (127.0.0.1, 80)
send: 'GET /~jcunningham/docs/.bzr/branch-format 
HTTP/1.1\r\nAccept-Encoding: identity\r\nConnection: 
Keep-Alive\r\nAccept: */*\r\nUser-agent: bzr/0.15.0dev0 
(urllib)\r\nHost: 127.0.0.1\r\nPragma: no-cache\r\nCache-control: 
max-age=0\r\n\r\n'
Request sent: [<bzrlib.transport.http._urllib2_wrappers.Request instance 
at 0xb7bb402c>]
reply: 'HTTP/1.1 401 Authorization Required\r\n'
header: Date: Wed, 28 Feb 2007 21:29:57 GMT
header: Server: Apache
header: WWW-Authenticate: Basic realm="sarat"
header: Content-Length: 460
header: Keep-Alive: timeout=15, max=100
header: Connection: Keep-Alive
header: Content-Type: text/html; charset=iso-8859-1
For status: [401], will ready body, length:  [460]
Consumed body: [<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
<html><head>
<title>401 Authorization Required</title>
</head><body>
<h1>Authorization Required</h1>
<p>This server could not verify that you
are authorized to access the document
requested.  Either you supplied the wrong
credentials (e.g., bad password), or your
browser doesn't understand how to supply
the credentials required.</p>
<hr>
<address>Apache Server at 127.0.0.1 Port 80</address>
</body></html>
]
Receives response: <bzrlib.transport.http._urllib2_wrappers.Response 
instance at 0xb76f9c2c>
   For: 'GET'('http://127.0.0.1/~jcunningham/docs/.bzr/branch-format')
Create addinfourl: <addinfourl at -1217422356 whose fp = 
<socket._fileobject object at 0xb7aadaac>>
   For: 'GET'('http://127.0.0.1/~jcunningham/docs/.bzr/branch-format')
bzr: ERROR: Invalid http response for 
http://127.0.0.1/~jcunningham/docs/.bzr/branch-format: Unable to handle 
http code 401: Authorization Required

Hmmm. Not sure what to make of that.
> 
>        Vincent
> 
> P.S.: It's a bit late here but I may find some more time tomorrow
> at worst.

Well, you have a good evening in any event.

Regards,
--Jeff