launchpad proxy support

[Gordon Tyler]

Vincent Ladeuil wrote:
> So a couple of general remarks: what you really want to reuse is
> the HTTPConnection objects in _urllib2_wrappers.
> 
> SO forget about the http_transport file entirely (we'll need some
> copyright assignments anyway so let's keep it simple too).
> 
> The idea is that you define some HTTP/HTTPS classes inside
> lp_registration and override their _connection_class attributes
> with the _urllib2_wrappers HTTPConection objects.  Doing that
> will avoid having to redefine the request method too.

I don't think just re-using the _urllib2_wrappers HTTPConection is going 
to do it. It needs the whole stack of the ProxyHandlers, 
RedirectHandlers, etc. that come from using a urllib2 URLOpener 
(appropriately modified by the wrapper).

However, if it makes you more comfortable, I can rewrite the 
HTTPTransport class into lp_registration.py. It bears very little 
resemblance to the original file posted by Bill on the Python bug 
report. I have rewritten it from scratch, pretty much.

I've just recently pushed another change to lp:~doxxx/bzr/lp_proxy to 
fix the underlying problem in the way urllib2 handles proxying 
connections to ports other than 80 (i.e. https' 443). I had to apply a 
workaround/hack to _urllib2_wrappers' Request and _ConnectRequest so 
that they could remember the original URL's port and supply that in the 
CONNECT request to the proxy.

This combined with using a custom xmlrpclib Transport to use 
_urllib2_wrappers to make the HTTP requests appears to have fixed the 
problem completely for me.

> Then, we will need tests, which has been the main blocking point
> for me so far.... I think the simplest here is to go with some
> fake server with pre-canned responses tailored to one request
> where the proxied server has one response and the not proxied one
> another response just to ensure we reach the good one.
> 
> We'lll have to test for the various proxies combinations too
> http[s] -> http[s].

I'll have to look at how _urllib2_wrappers tests and launchpad tests are 
written.

Ciao,
Gordon