[UGLY HACK] Proof of concept multipart/byteranges support and connection sharing

[Robert Collins]

On Mon, 2006-05-22 at 11:15 +1000, Michael Ellerman wrote:
> On Sat, 2006-05-20 at 20:57 +1000, Robert Collins wrote:


> Yeah, the Curl() stuff is purely to do connection sharing. I don't know
> how the transport code works, but from a quick glance, if we create a
> new HttpClient() for each get_transport(..) isn't that equivalent to
> having one Curl() per PyCurlTransport() ? If so, that doesn't get us as
> big a win, because we create lots of transports.

not at all - we typically use 'clone()' to 'chdir' around on a server.
get_transport is only called when we go from URL to Transport.

> As another data point, doing the byterange stuff without the connection
> sharing gets me these rough numbers:
> 
> real    29m8.436s
> real    31m13.397s
> real    29m5.429s
> 
> So it's definitely helping, although the bulk of the improvement is the
> byterange stuff.
> 
> > Just having good range support would rock. Its been on my TODO for a
> > bit. Things to watch out for in the multipart response: web servers may
> > return the full object, or may return combined ranges - IIRC the bytes
> > cant be reordered from the requested range though. You'll need to check
> > rfc2616 on that.
> 
> Yeah, there's lots of corner cases. I think I already handle the full
> result case, as long as we get code 200 back, not 206. Reordering would
> break that code in a jiffie. Before writing a proper version I'd like to
> check what twisted do, and/or any other implementations.

I have only seen one complete client implementation of ranges in source,
and thats in squid. My advice - go with the rfc :).

Rob
-- 
GPG key available at: <http://www.robertcollins.net/keys.txt>.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 191 bytes
Desc: This is a digitally signed message part
Url : https://lists.ubuntu.com/archives/bazaar/attachments/20060522/b174e8e8/attachment.pgp