why sftp push is so slow..

[Robert Collins]

On Sat, 2005-12-17 at 08:18 -0600, John Arbash Meinel wrote:

> I don't think just downloading the prelude will gain us a whole lot. It
> might, but the average weave size for bzr.dev is 8kB. Which means to get
> just the prelude, you have to read in chunks of about 1kB. Which I'm
> guessing will go back into hurting performance when you do have to copy
> something. Also, the default paramiko buffer size for reading is 8kB.
> 
> I'm not saying it wouldn't help at all. But I don't think we want to
> read line by line either.
> 
> I think the bigger help would be to go ahead and read the remote weave,
> and fix it up completely, with all revisions we have locally, rather
> than just the current one.
> Then if you are pushing 80 revisions, it doesn't have to download
> everything, add 1, and upload everything 80 times.
> Without caching, we would still be downloading 8kB, and at least the
> prelude for each weave.
> But if we just cache the prelude for as long as we hold the remote write
> lock, I think we could have a drastic performance improvement.

We already merge the weaves, so the entire file id is merged at once no
matter how many revisions we are pushing.

(see fetch.py: _copy_one_weave).

Though I've just noted a bug there: the if text_revision in to_weave
'get out quick clause' has the result of 80 roundtrips if every revision
is present. So indeed we can reorganise this a little to be more
efficient. Also the inventory weave is treated badly in this respect
too.

Lets get the file_involved_in optimisation in first, because I think it
has much the same net effect.



Rob

-- 
GPG key available at: <http://www.robertcollins.net/keys.txt>.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: This is a digitally signed message part
Url : https://lists.ubuntu.com/archives/bazaar/attachments/20051218/41ce9539/attachment.pgp