[RFC] Multipart support for _urllib_

[Michael Ellerman]

On 6/17/06, Michael Ellerman <michael at ellerman.id.au> wrote:
> Hi guys,
>
> I've cleaned up the work I did to do multipart HTTP. Actually I
> started from scratch, I think it's reasonably readable now.

As an FYI, while testing this I got curious about where our bottle neck is.

With my code I can branch bzr.dev from bazaar-vcs.org in around 21
minutes. That's somewhere around 30 MB of data.

I can pull the inventory.knit (7MB) in ~40s. Which would suggests I
could pull a 30MB blob in about 170 seconds, or a bit under 3 minutes.

If I capture all the GETs we're doing and ask wget to do them instead
(with -i, so one connection), it takes ~11 minutes.

So we could still do a bunch of work to speed up branching, but it
seems that having to do so many individual requests imposes a limit on
us.

cheers