[BUNDLE] let PyCurl transport read multiple ranges.

[Johan Rydberg]

"Michael Ellerman" <michael at ellerman.id.au> writes:

>> The differences between this patch and Ellermans are minimal;
>>
>>  * I've wrapped pycurl.Curl objects in a Curl object.
>>  * Transport.clone() reuses the Curl-object from the base transport.
>>  * Only one Curl-object is used per transport.
>>  * Ranges are combined (i.e., 10-20,20-30 is combined to 10-30.)
>
> Damn, we just had a mid air collision :)

Hehe.

> I've actually been quietly reworking this code off in a corner, it's
> not quite ready to go, there's no tests yet :}, but it's coming along.
> YMMV.
>
> Latest code is at: http://michael.ellerman.id.au/bzr/branches/http
>
> I'd still like to do something with caching the Curl objects, so when
> I get my stuff finished I'll try and merge your work to do that on
> top. And also the combining of ranges is cool for various reasons.

Maybe we should go with my patch for now, and when your stuff is
ready, we apply it?  Better to get some functionality in now than wait
forever for the "perfect patch" :)


> Another thing we really need to do is some sort of detection of how
> large a range header a server can handle. It looks like the default
> size in Apache2 is ~8K per header (ie. the range string has to be <
> than that), but it's configurable _downward_ by users. So it's quite
> possible we'll hit servers with 4K or 2K or less.

I tried sending 2000 ranges instead of 500, but actually lost
performance for some reason.  Did not investigate it further though.


~j