[storm] Weird 'pickle' related error when adding many objects

[Justin Warren]

Hi folks,

I've hit a weird problem that is beyond me at the moment. Hopefully
someone else can shed some light on the situation.

I create a bunch of objects, and attempt to add them all to the
database. If I only add about 100 objects, everything works fine. If I
attempt to add 150, I get a bunch of these errors:

Exception exceptions.AttributeError: AttributeError("'NoneType' object
has no attribute 'dumps'",) in <bound method
ObjectInfo._emit_object_deleted of {'store': <storm.store.Store object
at 0x139cc90>, 'primary_vars': (<storm.variables.IntVariable object at
0x1535910>, <storm.variables.DateTimeVariable object at 0x13bcc68>)}

'dumps' appears to only be used in variables.py as part of
pickle.dumps() calls. I added some print statements to get an idea of
what's going on, and narrowed the issue to ListVariable.get_state(). It
seems that pickle == None within this method, even though the 'import
cPickle as pickle' statement at the start of the file works to begin
with.

I've checked my environment, and I can indeed import cPickle. I added an
import statement just before the problem line, thus:

    def get_state(self):
        try:
            import cPickle as pickle
            return (self._lazy_value, pickle.dumps(self._value, -1))
        except AttributeError, e:
            print "erk! found the problem here."
            print "pickle is:", pickle

And then I get this:
Exception exceptions.ImportError: ImportError('No module named
cPickle',) in <bound method ObjectInfo._emit_object_deleted of {'store':
<storm.store.Store object at 0x139bc90>, 'primary_vars':
(<storm.variables.IntVariable object at 0x151c850>,
<storm.variables.DateTimeVariable object at 0x13a8fc8>)}>

It seems something is resetting pickle to None, as well as sys.path.
This seems really weird to me.

I'm using Thomas's twisted integration branch r214, and I've seen this
same issue in r207. It might be that the twisted integration is breaking
something, but I've searched for 'pickle' in all the code files and
found nothing. Nor can I find anything obviously updating sys.path.

Any clues, hints or help would be greatly appreciated.

-- 
Justin Warren <daedalus at eigenmagic.com>