gc.garbage from paramiko

[Robey Pointer]

On 7 Feb 2006, at 17:17, Andrew Bennetts wrote:

> On Tue, Feb 07, 2006 at 10:39:27AM -0800, Robey Pointer wrote:
> [...]
>>>
>>> What do you need to do on __del__?  socket.sockets already close
>>> themselves just
>>> like files.
>>
>> I guess that was kind of vague, sorry. :)  Python borrows a concept
>> from c++, and causes files & sockets to auto-close themselves as soon
>> as the last reference goes away.  Since paramiko supplies both  
>> socket-
>> like things (Channels) and file-like things (SFTPFiles), I've found
>> that python coders expect them to have the same del behavior as
>> builtins, allowing things like this:
>>
>>     open('email.txt', 'w').write('jdoe at example.com')
>
> But why do you need to do anything explicitly in paramiko to  
> support this?  The
> only external resource you have (that I'm aware of) is a socket,  
> and python's
> socket type already takes care of closing that when the last  
> reference goes
> away.

Right.  The problem comes when SFTPFile objects (for example) are  
used the same way.  The sftp session may hang around for a while, but  
the coder who uses the idiom above expects the file to be flushed and  
closed as soon as the "file object" goes out of scope.

BTW, I agree with those earlier in the thread who said this idiom is  
playing a little fast & loose.  But it's used a lot even in the bzr  
source, so don't throw too many stones. ;)


> So, I guess I'm missing what finalisation behaviour you want to  
> happen that
> doesn't already happen without __del__?  Sending some sort of  
> polite "quit"
> message to the server rather than just closing the socket?

In some cases, the file may not have finished writing its data yet  
until del'd or closed.  This is to sidestep latency issues when  
pushing a lot of data.

I know some of this may seem arcane & obfuscated.  I think it's  
important to take complex things and present them through a simple  
interface, so I go through some hoops to make ssh channels behave as  
much like sockets as possible (even to the point where you can select 
() on them), etc, even though it often makes the underlying  
implementation harder.


> In some cases it can be possible to replace __del__ with weakref  
> callbacks
> (which don't have this problem), although I can't say whether that's a
> possibility here without understanding what you're using __del__  
> for :)

[omitted clever tricks which I saved because they're pretty cool]

It doesn't look like I can use either the weakref callback (which I  
didn't know about till today) or the trick of adding a __del__ to a  
member variable, because in the destructor I'll need to effectively  
call "close()" on the destroyed object, which won't be available from  
either place.


>>> Cycles are really, really hard to avoid.  If a traceback occurs,
>>> that can create
>>> cycles.  There's lots of ways to unexpectedly make a cycle.
>>
>> How could a traceback create a cycle?
>
> See the warning at the top of http://www.python.org/doc/lib/inspect- 
> stack.html
>
> A traceback raised from inside paramiko and caught by a caller  
> might be used in
> all sorts of ways that paramiko can't know about -- including  
> keeping a
> reference to it (for later inspection, perhaps).  So even though  
> paramiko itself
> might not be creating cycles that it won't break, callers of  
> paramiko can
> easily make other cycles with paramiko inadvertently.

I see how that could make paramiko objects have a longer lifetime  
than expected, but not how it could create cycles.  Assuming I kept  
references in paramiko from having cycles (ie, made them a DAG),  
outside sources should be able to make references to anything inside,  
without breaking the DAG.

Sorry if this was too long/geeky.  I tried to go back and edit it down.

robey