Rev 3027: Rewrite http_readv to allow several GET requests. Smoke tested against branch reported in the bug. in file:///v/home/vila/src/bzr/bugs/165061/

John Arbash Meinel john at arbash-meinel.com
Thu Nov 29 16:17:59 GMT 2007


-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Vincent Ladeuil wrote:
> At file:///v/home/vila/src/bzr/bugs/165061/
> 

Looking at your implementation, I would say it would be a good time to bring
back _max_readv_combine = 50.

Basically, we end up doing a GET for every 50*200 raw ranges, and then start
returning the data for that range, and then going on to the next.

- From the original bug, (which is where @50 we had 671 compact ranges) that
would give us 4 GET requests. Which means 1/4th the amount of buffering before
we start returning stuff.

I think having an incremental HTTP parser would be wonderful. But if we can get
a large portion of that by just changing

- - _max_readv_combine = 0
+ _max_readv_combine = 50

I think there is a *huge* benefit versus effort tradeoff to consider.

Heck, I would even consider dropping it to _max_readv_combine = 10. That gives
us 2000 raw ranges per round trip to process. So when getting all of bzr.dev,
you will have 40 round trips. It is a bit much, but I think the process itself
will *feel* much snappier.

We won't really drop the wall clock time until we have async processing. So
that data can be buffering up in the socket while we are processing the earlier
stuff. That requires rewriting the http code, or changing your new code to
issue the next request before processing the current set of data.

This code doesn't look right to me, by the way:
+        for group in xrange(0, len(coalesced), max_ranges):
+            ranges = coalesced[group * max_ranges:group+1 * max_ranges]

At a minimum it should be:

ranges = coalesced[group * max_ranges:(group+1)*max_ranges]

But I think you are also running into a Robert-ism, where he gets the
parameters to xrange a bit off. And what you really want is:
+        for group in xrange(0, len(coalesced), max_ranges):
+            ranges = coalesced[group:group+max_ranges]

John
=:->
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.7 (Darwin)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFHTuY2JdeBCYSNAAMRAspwAKDSy6vKvKzPsfDMoNx4j9NXbzmUygCgqaCP
YAHlb2py+xigpVQm97YNDpE=
=iHR+
-----END PGP SIGNATURE-----



More information about the bazaar mailing list