[storm] suspected bug in caching behavior

[James Henstridge]

On Tue, Nov 11, 2008 at 9:10 PM, Andreas Kopecky
<andreas.kopecky at meduniwien.ac.at> wrote:
> James Henstridge wrote:
>>
>> That is one of the revisions that was merged in r271 of Storm's trunk.
>>  The bazaar.launchpad.net link shows the related changes that were
>> merged with it, and has a "diff" link at the top.
>
> ah ok. thank you.
>
>
>> Are you using multiple connections/stores to produce this behaviour?
>> The test case for this problem used 2 stores, but it'd be interesting
>> to know if the problem can be reproduced with a single store.
>
> The problem can definitly be reproduced with only 1 Store but using a
> Zope instance with more then 1 thread. Therefor it is rather tricky
> to write a testcase which produces this behavior consistently...

Assuming you're using the ZStorm utility, you'd be using one store per
thread so it would be a case of multiple stores.  So this definitely
sounds like the same bug.


>> Not clearing the alive list is intended behaviour.  The
>> checkpoint_state bit obviously wasn't :)
>
> I see. I was wondering about the fact that _load_object uses
> Store._alive to fetch cached objects rather then using Store._cache
> though. If it's intended behavior thats ok ofcourse - just wanted
> to suggest possible means of fixing it

The two collections serve different purposes.  The alive list keeps
track of every instance associated with the store but doesn't hold a
strong reference to them so they can be collected as normal.  The
cache list holds strong references to the most recently used objects
on the assumption that they might be used again soon.  The alive list
will hold everything in the cache list (as those objects are strongly
referenced) plus any other objects that haven't been collected.

If _load_object() used the cache list instead, then it would be
possible to create two instances representing the same row if the
first instance dropped out of the cache but was still referenced.
This would cause problems.


> Btw: i did not get around to test the fix you sent me yet due to being
> a bit busy with other things - i will write back (tomorrow at latest) if
> it worked.

That's cool.


>> That's nice to hear.  If you have ideas for improvements to the Zope
>> integration API, please tell us (either on the list or the bug
>> tracker).
>
> Improvements... couldn't think of anything that didn't work
> straightforward really. I did however write a Zope Container
> implementation that simulates Containers as are common in ZODB based
> on a relational database (i know that most say this is not a good idea
> ... but it really wasn't hard at all an allows to reuse most - i'd
> say 95% - of Zope base architecture)... If you are interested in such
> i'd happily send it around to you :)

I don't think it fits into the architecture of any of the apps we've
been building, but it does sound like this might be useful to others.
Do you think it is the kind of thing that would make sense to
distribute as part of Storm?

James.