[rfc] [patch] pycurl transport

[John A Meinel]

Martin Pool wrote:
> This patch adds a new http/https transport which uses the PyCurl
> library.  <pycurl.sf.net>
> 
> This has a few advantages over the builtin urllib:
> 
>  - it will use keepalive connections where possible
>  - it caches DNS entries
>  - it sets 'cache: must-revalidate' which should be the right
>    tradeoff between correctness and caching
> 
> The main disadvantage is that pycurl adds another dependency, and a
> dependency on a C library and its Python binding.  Given that I don't
> know if it should be the only option but perhaps it should be used if
> PyCurl can be found?
> 
> For me this cut transfer times from the UK to Australia by about 50%.
> There are probably bigger wins possible in other layers (e.g. Johan's
> revfile) but this should be compatible with them.
> 
> Any thoughts?
> 

What about gzip encoding. Weaves compress quite well. And it could cause
as much as a 4x download speed increase.

But I agree with Jamie. If you get as much of an increase as you say, it
is worth using if available.
Kind of like cElementTree, only not as necessary.

Oh, and it would be nice if you could actually use a HEAD request for
'has', and do a partial download for a get_partial request.

Though I'm not sure how it compares with the knit changes to transport.
Since they were switching to returning file-like objects that support
seek(). (Which I think makes it difficult to do a partial read, unless
all file-like objects return by Transport would also support scatter read).

John
=:->


-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 249 bytes
Desc: OpenPGP digital signature
Url : https://lists.ubuntu.com/archives/bazaar/attachments/20060110/8ba2af6e/attachment.pgp