[storm] using twisted-storm

[James Henstridge]

On Sat, Apr 18, 2009 at 6:35 AM, Lee Connell <lee.a.connell at gmail.com> wrote:
> Does anyone have any examples on how to use storm-twisted integration within
> a twisted server?
>
> Even an example on how to use storm in a blocking manner within a twisted
> server, from what I've read I should use deferToThread for any calls to the
> store, is this still the best way to do this?
>
> Any examples/info you can provide to get me started would be great, thanks!

I haven't played with the twisted-integration branch, but we have used
Storm's normal API in some twisted apps in a blocking fashion.  The
basic parts are:

1. Use the storm.zope code to manage per-thread stores and global
transaction management.  This requires Zope's transaction and
zope.interface packages.  If you're using Twisted, then you should
already have zope.interface though.

2. Configure the stores you want to use:
     from storm.zope.zstorm import global_zstorm
     global_zstorm.set_default_uri('whatever', 'postgres:...')

3. When you want to perform any database work, call a function with
deferToThread.

4. The database function should begin a transaction on start and
commit the transaction on exit (or abort if you're only doing read
only stuff).  A decorator is a good way to handle this.  Something
like:

    import transaction
    from twisted.python.util import mergeFunctionMetadata

    def write_transaction(function):
        def wrapper(*args, **kwargs):
            transaction.begin()
            try:
                result = function(*args, **kwargs)
            except:
                transaction.abort()
                raise
            else:
                transaction.commit()
                return result
        return mergeFunctionMetadata(function, wrapper)

5. The database function can get the store with
global_zstorm.get('whatever').  No Storm objects should be returned by
the function.

That's about it.  The zstorm code will manage per-thread stores, and
deferToThread() will run your callbacks in a thread pool, so you
should reach a steady state number of connections for a long running
application.

James.