why sftp push is so slow..

[John Arbash Meinel]

Robert Collins wrote:
> when pushing a revision FOO, we check that every fileid at revisionid it
> references is present in the remote branch. This requires downloading
> all the weave contents at the moment. 
> 
> I suspect that in the interim before knits, a prelude based check that
> pulls only the remote prelude, compares that to the local weave about to
> be pushed, and only if needed pulls and pushes the weave, would make a
> huge difference.
> 
> Rob

I don't think just downloading the prelude will gain us a whole lot. It
might, but the average weave size for bzr.dev is 8kB. Which means to get
just the prelude, you have to read in chunks of about 1kB. Which I'm
guessing will go back into hurting performance when you do have to copy
something. Also, the default paramiko buffer size for reading is 8kB.

I'm not saying it wouldn't help at all. But I don't think we want to
read line by line either.

I think the bigger help would be to go ahead and read the remote weave,
and fix it up completely, with all revisions we have locally, rather
than just the current one.
Then if you are pushing 80 revisions, it doesn't have to download
everything, add 1, and upload everything 80 times.
Without caching, we would still be downloading 8kB, and at least the
prelude for each weave.
But if we just cache the prelude for as long as we hold the remote write
lock, I think we could have a drastic performance improvement.

John
=:->
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 256 bytes
Desc: OpenPGP digital signature
Url : https://lists.ubuntu.com/archives/bazaar/attachments/20051217/d376d105/attachment.pgp