[storm] why weakrefdict for cache?

[Gustavo Niemeyer]

Hey Mike!

> I've been watching this thread, and I thought I'd mention that in
> SQLAlchemy, we moved from a weak-referencing identity map to a strong
> referencing one by default (with a weak-valued option) many versions
> ago.  The use case on our side for the strong-referencing map is someone
> who is making changes to multiple objects but not maintaining them in
> the local namespace; since SA's Session currently does not maintain a
> strongly referenced collection of "dirty" objects, they'd get garbage
> collected before a flush could occur. An LRU-based approach wouldn't fix
> this for us.

Right.  In Storm this won't be an issue.  When objects get dirty they are
added to a dictionary which will strong-reference them so that they are
kept in memory at least up to the next flush/rollback.  They continue to
be in the weakrefd cache even then, and only leave that one when they die.

> While we might someday reinstate a strongly-referenced "dirty"
> collection, the basic idea of a strongly referenced identity map is
> generally not a problem for our users; the use case where someone is
> looping through many objects and throwing away as they iterate is
> pretty rare and those folks either expunge the objects explicitly or
> use the "weak referencing" option on their session.

I see.. but then do they have to make sure by themselves that the
object doesn't die before it gets flushed?

Did you consider the creation of a more flexible caching system, and
if so, can you tell me why you gave up?  (maybe there's something
we can learn from that)

I'm actually a bit surprised that people don't seem to bother with
the strong references for the duration of the transaction.
In Landscape, for instance, we have web pages which show up thousands
of objects, and there isn't a good reason to keep the object in
memory after it has been displayed.

Thanks for the feedback,

-- 
Gustavo Niemeyer
http://niemeyer.net