[storm] Weird 'pickle' related error when adding many objects

[Andreas Kopecky]

Hi Justin,

I tried to reproduce what you describe here but failed at it.
I think you need to give us a little more information on what kind
of objects you try to create, what fields they got and (if possible)
an easy to reproduce testscript.

i tried something along the lines of:

 >>> storage = []
 >>> for i in xrange(1000):
 >>>     obj = Test(u'testing name %i' % i)
 >>>     storage.append(obj)
 >>> print [store.add(obj).name for obj in storage]

whereas Test() is a very basic storm object - and it works like a charm

regards,
- Andreas Kopecky

Am 24.11.2008 um 01:32 schrieb Justin Warren:

> Hi folks,
>
> I've hit a weird problem that is beyond me at the moment. Hopefully
> someone else can shed some light on the situation.
>
> I create a bunch of objects, and attempt to add them all to the
> database. If I only add about 100 objects, everything works fine. If I
> attempt to add 150, I get a bunch of these errors:
>
> Exception exceptions.AttributeError: AttributeError("'NoneType' object
> has no attribute 'dumps'",) in <bound method
> ObjectInfo._emit_object_deleted of {'store': <storm.store.Store object
> at 0x139cc90>, 'primary_vars': (<storm.variables.IntVariable object at
> 0x1535910>, <storm.variables.DateTimeVariable object at 0x13bcc68>)}
>
> 'dumps' appears to only be used in variables.py as part of
> pickle.dumps() calls. I added some print statements to get an idea of
> what's going on, and narrowed the issue to ListVariable.get_state().  
> It
> seems that pickle == None within this method, even though the 'import
> cPickle as pickle' statement at the start of the file works to begin
> with.
>
> I've checked my environment, and I can indeed import cPickle. I  
> added an
> import statement just before the problem line, thus:
>
>    def get_state(self):
>        try:
>            import cPickle as pickle
>            return (self._lazy_value, pickle.dumps(self._value, -1))
>        except AttributeError, e:
>            print "erk! found the problem here."
>            print "pickle is:", pickle
>
> And then I get this:
> Exception exceptions.ImportError: ImportError('No module named
> cPickle',) in <bound method ObjectInfo._emit_object_deleted of  
> {'store':
> <storm.store.Store object at 0x139bc90>, 'primary_vars':
> (<storm.variables.IntVariable object at 0x151c850>,
> <storm.variables.DateTimeVariable object at 0x13a8fc8>)}>
>
> It seems something is resetting pickle to None, as well as sys.path.
> This seems really weird to me.
>
> I'm using Thomas's twisted integration branch r214, and I've seen this
> same issue in r207. It might be that the twisted integration is  
> breaking
> something, but I've searched for 'pickle' in all the code files and
> found nothing. Nor can I find anything obviously updating sys.path.
>
> Any clues, hints or help would be greatly appreciated.
>
> -- 
> Justin Warren <daedalus at eigenmagic.com>
>
>
> -- 
> storm mailing list
> storm at lists.canonical.com
> Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/storm
>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: https://lists.ubuntu.com/archives/storm/attachments/20081124/25b3c7a5/attachment.htm 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: PGP.sig
Type: application/pgp-signature
Size: 486 bytes
Desc: Signierter Teil der Nachricht
Url : https://lists.ubuntu.com/archives/storm/attachments/20081124/25b3c7a5/attachment.pgp