[storm] Compressed Pickle and RawStrings

[Ryan Haynes]

Hello All,

I recently started using storm.  Thanks a lot for the work on this.  A
quick question... I have relatively sparse numpy objects that I'd like
to persist into RawStr.  Would it make sense to add a compression
facility into the Columns that accept blobs?  I currently just do the
following...

class BlockAttribution(Block):
    __storm_table__ = "block_attributions"
    #  This is a bz2 compressed pickled numpy array.
    _data = None
    _data_db = RawStr(name=u"data")

    @property
    def data(self):
        if self._data_db is None:
            return None
        elif self._data is None:
            self._data = pickle.loads(bz2.decompress(self._data_db))
        return self._data

    @data.setter
    def data(self, value):
        self._data = value
        self._pack(value)

    def _pack(self, value):
        compress_lvl = 5
        self._data_db = bz2.compress(pickle.dumps(self._data), compress_lvl)


    @data.deleter
    def data(self):
        self._pack(self._data)
        del self._data

It seems to persist to the media well.  However, this is only safe
because I'm working in a non-threaded context and I expect all changes
to this object to local until I flush it to the DB.

I dunno, maybe this is a completely broken thing to do.  But if there
was something like a opencl BZ library this could be an awesome way of
persisting objects.

Cheers,
Ryan