[storm] Pooling ?

[Gustavo Niemeyer]

Hey Andy,

> I am working with CherryPy and am trying to figure out the best way

That's very cool!  I'm looking forward to see it being seamlessly
integrated in CherryPy.

> to incorporate Storm into it. Does Storm have any connection pooling?  
> If not then what is the best way to handle ensuring the connections  
> are still alive with each thread?  I would assume we would have to  
> write something that would do a check/auto-connect if the threads  
> connection was no longer alive. How should that be handled?

We actually implement a slightly different pooling mechanism, which
was recently improved by the landing of a patch from James Henstridge.

I'll explain how we do it in Zope3, and how the "pooling" system works,
so that you can imagine how to integrate it in CherryPy.

Here is how it works, from a high-level perspective: we have a ZStorm
utility which provides per-thread Store instances based on a name=>uri
mapping (code asks for zstorm.get("main-store"), for instance).  This
utility is actually very simple, except for the logic to hook it in
Zope's transaction mechanism, which you don't need.

So how would pooling work in that environment?  Well, we actually
don't need a "traditional" pooling system, because each thread has its
own Store, which is kept alive across transactions and requests.

The last step missing in that environment is how to handle disconnections,
and that's what James' patch implements (merged, but not yet released).
If a disconnection happens, an error is raised, but Storm will
automatically try to reconnect when the next transaction starts (e.g.
after a rollback()).

I guess the easiest would be for you to have a look at the zstorm code
(in storm/zope/*), and check it out how it works, probably even get some
code from there to an equivalent system for CherryPy.

-- 
Gustavo Niemeyer
http://niemeyer.net