attn robey - paramiko readv [again]
Robert Collins
robertc at robertcollins.net
Thu Apr 20 00:04:45 BST 2006
On Wed, 2006-04-19 at 15:27 -0700, Robey Pointer wrote:
> On 19 Apr 2006, at 6:18, Robert Collins wrote:
>
> > Hi Robey,
> > I'm working on making knits the default in the test suite, and I'm
> > finding a set of lockups I can only put down to race conditions with
> > readv: if I step into the readv routine, it works, if I call it, it
> > locks up in recv in this thread.
> >
> > Would a newer paramiko (using 1.5.2 at the moment) help ? Or is it a
> > StubServer issue ?
>
> You can't be using 1.5.2! It didn't have readv. :)
We still have seek + read in the sftp transport code.
> Try 1.5.4 -- even though it was in 1.5.3 it wasn't quite working
> right (and was undocumented).
bzr HEAD fixes the race I was reporting, but adds a new one:
File
"/home/robertc/source/baz/integration/bzrlib/tests/test_permissions.py",
line 224, in test_new_files
b_sftp.pull(b_local)
File "/home/robertc/source/baz/integration/bzrlib/decorators.py", line
51, in write_locked
return unbound(self, *args, **kwargs)
File "/home/robertc/source/baz/integration/bzrlib/branch.py", line
1188, in pull
return super(BzrBranch5, self).pull(source, overwrite,
stop_revision)
File "/home/robertc/source/baz/integration/bzrlib/decorators.py", line
51, in write_locked
return unbound(self, *args, **kwargs)
File "/home/robertc/source/baz/integration/bzrlib/branch.py", line
1110, in pull
old_count = len(self.revision_history())
File "/home/robertc/source/baz/integration/bzrlib/decorators.py", line
38, in read_locked
return unbound(self, *args, **kwargs)
File "/home/robertc/source/baz/integration/bzrlib/branch.py", line
1024, in revision_history
history = [l.rstrip('\r\n') for l in
File "/home/robertc/source/baz/integration/bzrlib/decorators.py", line
38, in read_locked
return unbound(self, *args, **kwargs)
File "/home/robertc/source/baz/integration/bzrlib/lockable_files.py",
line 176, in get_utf8
return codecs.getreader('utf-8')(self._transport.get(relpath))
File "/home/robertc/source/baz/integration/bzrlib/transport/sftp.py",
line 382, in get
f.prefetch()
File "/home/robertc/source/baz/integration/paramiko/sftp_file.py",
line 365, in prefetch
self._start_prefetch(chunks)
File "/home/robertc/source/baz/integration/paramiko/sftp_file.py",
line 408, in _start_prefetch
self._prefetch_so_far = chunks[0][0]
IndexError: list index out of range
I'll try using readv directly in paramiko now.
> I've made one tweak since then that's only in the bzr branch*, but it
> would only affect rapidly doing multiple readv's on the same file, or
> a prefetch followed immediately by a readv (readv is implemented as a
> form of prefetch). Is it possible that's what's happening? If so,
> the bzr branch should fix it.
>
> [But if you do a prefetch followed-by readv, you will get terrible
> performance anyway. Maybe I should have readv notice that situation
> and just use the prefetched data in that case. Hmm.]
>
> I'll need to make a 1.5.5 soon anyway, because Alexander helped me
> find a bunch of Windows issues in the sftp unit tests.
Cool. We'll want to get a stable combination in dapper :).
Rob
--
GPG key available at: <http://www.robertcollins.net/keys.txt>.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 191 bytes
Desc: This is a digitally signed message part
Url : https://lists.ubuntu.com/archives/bazaar/attachments/20060420/83ed4b24/attachment.pgp
More information about the bazaar
mailing list