attn robey is atomic sftp append possible?

[Robey Pointer]

On 27 Feb 2006, at 23:17, Robert Collins wrote:

> Robey, I want to make the contract for transport.append be that it
> returns the file length where the append started, and that the  
> appended
> data is written consequetively.
>
> Is that feasible give the sftp protocol?

They're actually several different questions so I'll give you several  
different answers. :)

1. Are sftp writes atomic?

     "Maybe."  Which means you can't count on it.  As usual, the  
version everyone has implemented (v3) promises nothing, while the  
version in committee (v6 so far) promises the world with lots of hand- 
waving.  But you can pack nearly 32KB of data into a single sftp  
packet, and I can't imagine that a respectable server would implement  
that as anything other than a single write() syscall.

2. Can you get the file length on append?

     A quick test of open(filename, 'a').tell() indicates that this  
should work.

3. Is the appended data written consecutively?

     I'm assuming you mean "can other writes interleave?"  Oddly,  
that might be possible in append mode, since sftp implies that it  
will ignore the "offset" argument to writes when in append mode.   
Normally sftp writes are equivalent to pwrite(): each write has an  
explicit position.  However I think multiple writers will be  
forbidden by our locking system so maybe it doesn't matter.

I think by knowing there can only be one writer at a time, the sftp  
transport can promise: to provide the offset where it started  
writing, and that on success no interleaving data snuck in.  It can't  
promise that a reader won't see the partial results of a write, and I  
suspect that gets more likely as the amount of data to write gets  
larger.

I hope that helps / makes sense.

robey