[MERGE] 1.6 fetch regression

[John Arbash Meinel]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Sending again because I forgot to say "merge".

John Arbash Meinel wrote:
> John Arbash Meinel wrote:
> ...
>
>> My best guess is that we are getting some really long buffered bytes, and the
>> cost of reallocating that is costing us a lot.

This ended up being a (the?) major factor.
Specifically, in the StatefulDecoder we would buffer up the bytes by doing:

self._in_bytes += bytes

Which would cause us to reallocate large (MB+) strings.

And then in the readv parser, we would do:

 data = data[consumed:]

Which would cause us to downsize the strings. Notice, though, that it requires
2x memory while doing it, because it has to create a new string, copy the
bytes over, and then destroy the original.

My initial work on fixing the "input" side failed, because we peek many times
at the string while waiting for more bytes to come. But I've since worked
around that.

The attached patch drops the time for "bzr branch bzr+ssh://localhost/bzr.dev"
from 1m26s to 39s.

$ old
VmPeak:	  202736 kB
bzr -Dmemory branch  xxx  54.60s user 30.11s system 96% cpu 1:27.58 total
$ new
VmPeak:	  181136 kB
 33.64s user 0.84s system 91% cpu 37.640 total

Notice the *huge* difference in "system" time.

John
=:->

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFItx2eJdeBCYSNAAMRAsm7AKCfs3yshkZwvo7cGogfgHqT208FygCfRU/d
I9Fq1ebYI30R05vgpKtGSK0=
=w2uJ
-----END PGP SIGNATURE-----
-------------- next part --------------
A non-text attachment was scrubbed...
Name: hpss_readv.patch
Type: text/x-diff
Size: 21038 bytes
Desc: not available
Url : https://lists.ubuntu.com/archives/bazaar/attachments/20080828/1405425f/attachment.bin