[RFC] Multipart support for _urllib_

[John Arbash Meinel]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Michael Ellerman wrote:
> Hi guys,
> 
> I've cleaned up the work I did to do multipart HTTP. Actually I
> started from scratch, I think it's reasonably readable now.
> 
> For the moment it doesn't handle the case of sending too many ranges
> to the server. Now that we sort the offsets, we get much better
> combining. I'm seeing at most 16 ranges get sent, which should be ok
> for most servers you'd hope. To be robust we should probably support
> falling back to a full request if we get a 400 for a range request.
> 
> Currently I've hooked it into urllib, because I can't for the life of
> me work out how to get the "Content-range" header out of pycurl. I
> don't think it can be done in fact. So hooking this up to pycurl will
> require a kludge, if we get back a non-multipart 206 we'll just have
> to assume it contains the union of all the ranges we requested plus
> any gaps.

I figured it out. It turns out that 'pycurl' actually has a fairly nice
wrapper available if you import the 'curl' package. pycurl is the C
extension, 'curl' is a wrapper around that.

Anyway you can use 'setopt(HEADERFUNCTION, sio.write)' which will let
you read in the header values.
On top of that, you can then use the mimetools package to parse
everything into a dictionary like object.
This makes it quite nice.

I haven't figured out exactly how to interface it with your changes, but
my http branch now supports getting any headers out of curl. (It
probably does too much right now, but I'll live with it for the moment).

http://bzr.arbash-meinel.com/branches/bzr/http/

John
=:->

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.0 (Darwin)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFElB5IJdeBCYSNAAMRArYSAJ4tNmmMvnLrYjWN1hXPenMn7jKqlACdFLvC
33SCp/A8vwnM4Lcd2uv5ry0=
=ne1Q
-----END PGP SIGNATURE-----