[PATCH] Re: Feature Request: 'bzr mv --after' to tell bzr that file(s) have already been moved in the working tree

[John Arbash Meinel]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

John Arbash Meinel wrote:
> Aaron Bentley wrote:
> ...
> 
>>>>>   - run 'bzr selftest', because the selftest of 'bzr.dev' seems to hang
>>>>> on my machine
>>> That's pretty serious.  Does "bzr selftest whoami" hang, or is it only
>>> long runs that hang?  Does a progress bar show?  Does it print anything?
>>>
>>> Aaron
> 
> I've been able to reproduce the hang.
> 

...

> 
> I'm going to try changing my version of paramiko, and see if that fixes
> some of this stuff (right now I'm using 1.6)
> 
> John
> =:->

I upgraded paramiko, and there are still a couple tests that fail, but
nothing hangs, which is good. (I'm using the latest tip from
http://www.lag.net/paramiko/bzr/paramiko/ which should be 1.6.3 plus a
couple of bugfix patches.

I do get a bunch of errors during cleanup that look like this:
Traceback (most recent call last):
  File "C:\Python24\lib\threading.py", line 442, in __bootstrap
  File "C:\Python24\lib\threading.py", line 422, in run
  File "H:\dev\bzr\bzr.dev\bzrlib\transport\sftp.py", line 1053, in
_run_server_entry
  File "H:\dev\bzr\bzr.dev\bzrlib\transport\sftp.py", line 1139, in
_run_server
  File "H:\dev\bzr\bzr.dev\bzrlib\transport\sftp.py", line 1053, in
_run_server_entry
  File "H:\dev\bzr\bzr.dev\bzrlib\transport\sftp.py", line 1139, in
_run_server
exceptions.AttributeError: 'NoneType' object has no attribute 'error'


Which means that our server thread isn't cleaning up at the right time,
because it is cleaning up during interpreter shutdown. We should
probably fix it to not cause errors during that time, as well as working
hard to make sure the thread gets stopped when we are done using it,
rather than hanging around until it is killed by interpreter shutdown.

The exception fixes can be done by doing something like the attached patch.

I'm not sure why the server thread isn't cleaning up in time. It may be
that it is scheduled to cleanup once it times out, which may be set for
30s. (So if we are just running the sftp tests, then the last test
hasn't had 5s before the test suite starts cleaning up)

The attached patch isn't complete. It fixes the direct bug, but then
causes a different one:
Traceback (most recent call last):
  File "C:\Python24\lib\threading.py", line 442, in __bootstrap
  File "C:\Python24\lib\threading.py", line 422, in run
  File "H:\dev\bzr\bzr.dev\bzrlib\transport\sftp.py", line 1053, in
_run_server_entry
  File "H:\dev\bzr\bzr.dev\bzrlib\transport\sftp.py", line 1155, in
_run_server
  File "C:\Python24\lib\threading.py", line 442, in __bootstrap
  File "C:\Python24\lib\threading.py", line 422, in run
  File "H:\dev\bzr\bzr.dev\bzrlib\transport\sftp.py", line 1053, in
_run_server_entry
  File "H:\dev\bzr\bzr.dev\bzrlib\transport\sftp.py", line 1155, in
_run_server
  File "c:\Python24\Lib\site-packages\paramiko\sftp_server.py", line
108, in finish_subsystem
exceptions.TypeError: super() argument 1 must be type, not None


So it seems that paramiko doesn't support cleaning up during process
shutdown. Considering it is using the current Class in that scope, it
seems that there really isn't anything we can do. What we *should* be
doing is making sure all threads have cleaned up before we let the
interpreter close.

What I really wish is that the sftp server process had a way to be told:
"Hey, I'm done with you, cleanup". Rather than our current methods of
letting socket.select() or whatever time out, and having the server
process clean up late.

The other possibility to make this happen is to change this line:
=== modified file 'bzrlib/transport/sftp.py'
- --- bzrlib/transport/sftp.py    2006-09-18 07:58:41 +0000
+++ bzrlib/transport/sftp.py    2006-11-06 16:49:34 +0000
@@ -1083,7 +1083,7 @@
         if sys.platform == 'win32':
             self._root = ''
         self._listener = SocketListener(self._run_server_entry)
- -        self._listener.setDaemon(True)
+        # self._listener.setDaemon(True)
         self._listener.start()


At the moment, this causes bzr to hang for a long time, (possibly
forever, I didn't really wait very long). Because the background threads
never realize that they are done working. I may look into a patch for that.

John
=:->


-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.5 (Cygwin)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFFT2gUJdeBCYSNAAMRAk77AKCY7ChWNjq99G7+05uwPthyBG9HaQCfXwnQ
FkjzIPtczxv0fAiBeCX0B9A=
=8PDn
-----END PGP SIGNATURE-----
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: sftp_cleanup.diff
Url: https://lists.ubuntu.com/archives/bazaar/attachments/20061106/04aecf61/attachment.diff