knits - slow sftp pushes

[John Arbash Meinel]

Robert Collins wrote:
> we pay versions per file * 2 * round-trip-time on push operations.
> This occurs because I overlooked it when optimising knits the first  
> time around. The mechanism is that we write each version as they 
> stream out of the source knit, which involves two writes: one to the
> data file and one to the index. One solution is to have the sftp 
> transport operate in a write-behind fashion and add a 'sync' or 
> 'flush' style call. I'm not in favour of this. Another approach is
> to buffer all the writes into a single big append to each of the two
> files inside the knit join operation. This is safer because it does
> not involve us writing to the index in advance of data actually 
> being confirmed by the server.
> 
> Thoughts? volunteers to do it?
> 
> Rob
> 

I think the join() should indeed work by doing buffering. I don't know
exactly how the code is working, but shouldn't join be doing everything
in memory, and then flushing to disk when it is done?

I believe that is how the old weave code worked where you would build up
your Weave, and then tell it to write to disk. But I didn't follow the
VersionedFile stuff enough to know if they kept that format.

John
=:->

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 254 bytes
Desc: OpenPGP digital signature
Url : https://lists.ubuntu.com/archives/bazaar/attachments/20060501/3a356b1a/attachment.pgp