[MERGE] A hack to make urllib not call recv(1) lots and lots.

[Andrew Bennetts]

John Arbash Meinel wrote:
[...]
> 
> Well, you could always use 'netem' to configure a high latency link on
> your own computer. I've used it before when testing stuff like 'bzr push'.
> 
> Add 50ms latency to the loopback (100ms round-trip ping time):
>  sudo tc qdisc add dev lo root netem delay 50ms
> 
> Restore it back to normal
>  sudo tc qdisc del dev lo root

Thanks!

I've used your recipe, and I've just repeated my experiments from last night.
I also followed Robert's suggestion, and bumped the buffering up to 64kB
(updated patch attached for completeness).

Here's unpatched bzr.dev, run twice:

real    2m48.769s
user    2m14.812s
sys     0m1.324s

real    2m46.799s
user    2m14.644s
sys     0m1.388s

And here's my hack:

real    2m39.456s
user    2m6.116s
sys     0m1.200s

real    2m38.222s
user    2m5.864s
sys     0m1.392s

So it appears it *does* make a measurable improvement.

FWIW, the branch I'm using can also be found at
http://twistedmatrix.com/~spiv/bzr/twisted/trunk/

-Andrew.