[BUNDLE] let PyCurl transport read multiple ranges.

[Michael Ellerman]

On 6/9/06, Johan Rydberg <jrydberg at gnu.org> wrote:
> "Michael Ellerman" <michael at ellerman.id.au> writes:
>
> >> The differences between this patch and Ellermans are minimal;
> >>
> >>  * I've wrapped pycurl.Curl objects in a Curl object.
> >>  * Transport.clone() reuses the Curl-object from the base transport.
> >>  * Only one Curl-object is used per transport.
> >>  * Ranges are combined (i.e., 10-20,20-30 is combined to 10-30.)
> >
> > Damn, we just had a mid air collision :)
>
> Hehe.
>
> > I've actually been quietly reworking this code off in a corner, it's
> > not quite ready to go, there's no tests yet :}, but it's coming along.
> > YMMV.
> >
> > Latest code is at: http://michael.ellerman.id.au/bzr/branches/http
> >
> > I'd still like to do something with caching the Curl objects, so when
> > I get my stuff finished I'll try and merge your work to do that on
> > top. And also the combining of ranges is cool for various reasons.
>
> Maybe we should go with my patch for now, and when your stuff is
> ready, we apply it?  Better to get some functionality in now than wait
> forever for the "perfect patch" :)

I'd rather not, that original code I wrote is _really_ terrible. I'll
have something testable soon that's not so ugly.

> > Another thing we really need to do is some sort of detection of how
> > large a range header a server can handle. It looks like the default
> > size in Apache2 is ~8K per header (ie. the range string has to be <
> > than that), but it's configurable _downward_ by users. So it's quite
> > possible we'll hit servers with 4K or 2K or less.
>
> I tried sending 2000 ranges instead of 500, but actually lost
> performance for some reason.  Did not investigate it further though.

Hmm, it probably ignored the range and sent you the whole file. Any
idea which server that was?

cheers