[storm] suspected bug in caching behavior

[James Henstridge]

On Tue, Nov 11, 2008 at 3:27 AM, Andreas Kopecky
<andreas.kopecky at meduniwien.ac.at> wrote:
> Hello storm mailing list,
>
> This is my first post here so please forgive me if this is
> a.) not the right place for this kind of message or
> b.) this mail is not in the right form
>
> Subject basically says it all - i think i encountered faulty/buggy
> behavior in the way Storm (or rather storm.store.Store) handles
> caching of ObjectInfo objects.
>
> Bug description:
>
> Updates to Storm object don't always reset the _checkpoint_state of
> this objects ObjectInfo.
>
> Reason:
>
> (Just a brief preface to make sure i get Storm mechanics right)
> Objects in a Storm Store are cached on two different levels. At first
> their ObjectInfo is stored in Store._cache, secondly a copy of this is
> also stored in Store._alive (a WeakValueDictionary).
>
> Whenever an object is loaded via Store.find, Store._load_object
> is called, which (if the object is loaded the first time)
> instantiates an ObjectInfo object and puts it into Store._cache and
> Store._alive (via Store._add_to_alive). Loading a cached object in
> Store.find - in contrary to what would be expected - does NOT ask
> Store._cache but rather Store._alive. When a transaction boundary
> is reached (on either Store.commit or Store.rollback) the cache
> is purged via Store.invalidate. HOWEVER, the Store._alive dict
> stays untouched until it is cleared by garbage collection.
>
> Result:
>
> When you load an object that has been cached previously and happen
> to be faster then the garbage collection you might get an artifact
> of an ObjectInfo object which is still in a state that is no longer
> in sync with the database.
>
> Proposed fixes:
>
> - clear Store._alive in Store._invalidate
> or
> - ask Store._cache if an object is - in fact - cached
>
> Some others are possible too but i guess you sure got more insight
> into Storm then i do :)

This sounds a bit like https://bugs.launchpad.net/storm/+bug/277095
which has been fixed on trunk since the 0.13 release.

If you have the time, would you mind trying to reproduce the bug with
the development version of Storm?  Alternatively, you could try
applying the changes from
http://bazaar.launchpad.net/~storm/storm/trunk/revision/271 to the
0.13 release and see if that helps.

James.