[storm] why weakrefdict for cache?

[Bernd Dorn]

On 03.09.2007, at 16:15, Gustavo Niemeyer wrote:

>
>> this is reasonable ... i just thought about that we might have zope
>> specific caching requirements, that might be common to zope
>> applications, such as ReferenceSet caching etc. But as you said, it
>> is the job of a pluggable caching mechanism.
>
> True.. we may have a pluggable cache for zstorm, if we find out that
> the the default isn't good for Zope applications.  Even then, it'd
> be good if we managed to find a caching system that does a good job
> for most cases, including ours (lovely + Landscape + whoever is
> watching :-).
>
>> When do you think will such someone find time to implement caching.
>> Or shall we (lovely) start to do this?
>
> There's a feature in Landscape which is urging my attention, so I'm
> unlikely to be able to look at this before some time next week. Here
> is an initial implementation plan, in case you want to tackle it:

for me it's impossible to before next week :-( but we will need this  
feature at the end of september, so if you can't find the time up  
till then, we gonna implement it.

>
> - Let's rename Store._cache to Store._alive.  That reflects better
>   what this attribute means.  We shouldn't have to touch the use of
>   _alive (caching will work side-by-side with the current system).
>
> - We create a cache implementation in cache.py.  It has a very
>   simple API: add(), remove(), and clear().  Store._cache will hold
>   it (let's not implement the "pluggable" aspect in the first
>   version, so that we have time to learn if our API is right).
>
> - Whenever an object is returned to the user from the Store
>   or from a ResultSet (no matter if it's loaded from database or
>   not), it's add()ed to the cache.
>
> - Internally, the cache maintains two structures: one dictionary
>   with {obj_info: obj} (this has the strongref to the obj), and
>   one list of obj_infos.  We can't have just a list of the
>   objects themselves because we don't want to touch any special
>   methods in the object implementation (e.g. __eq__).
>
> - If Cache.add() finds that obj_info is already in the dictionary,
>   it removes the object from the list, and reinserts it at the
>   start.  If the list goes above N, the last element is popped
>   from the list and the dictionary.
>
> - Store.invalidate() takes the item off the cache.  It should also
>   be taken off on _remove_from_alive (which is _remove_from_cache
>   right now, but will be renamed).
>
> - We need 100% test coverage for that, if possible using TDD.
>
>
> How does that look?
>

this looks very good!

what do you think about referenceset caching use-cases? not for the  
first iteration, but i would keep it in mind

> -- 
> Gustavo Niemeyer
> http://niemeyer.net