[storm] Storm and DBUtils

Sean Allen sean at ardishealth.com
Sat Mar 1 17:30:20 GMT 2008


On Mar 1, 2008, at 12:08 PM, Christopher Armstrong wrote:

> On Sat, Mar 1, 2008 at 11:58 AM, Sean Allen <sean at ardishealth.com>  
> wrote:
>>
>> On Feb 29, 2008, at 4:56 PM, Christopher Armstrong wrote:
>>
>>> On Fri, Feb 29, 2008 at 4:29 PM, Sean Allen <sean at ardishealth.com>
>>> wrote:
>>>> Before I start in on this, I wanted to see if anyone had done any
>>>> work
>>>> on integrating DBUtils and Storm.
>>>>
>>>> If not, away I go, but if someone has, please point me in the  
>>>> proper
>>>> direction.
>>>
>>> I'm not sure if it makes sense to integrate the two; Storm provides
>>> many of the features already. All you need to do is make sure that
>>> Store objects are not shared between threads. Reconnection is done  
>>> by
>>> Storm itself. Pooling is unnecessary. Managing thread-specific Store
>>> instances is something that's left up to you (or the framework), but
>>> we provide storm.zope to help with this (as well as zope transaction
>>> integration) for Zope users. It'd be great to add support for more
>>> thread-managing frameworks over time.
>>
>> I'm particular interested in the persistent connection aspect of
>> DBUtils which I cant find any implementation of in Storm.
>>
>> Am I wrong on that?
>
> The latest release of Storm automatically reconnects when the
> connection to the SQL server is lost. The statement that is being run
> at the time the connection is lost will fail (with an exception type
> which you can use to retry a transaction), but the next transaction
> will work.

What I'm interested in is not closing out the connections period.
If I read the DBUtils code correctly ( and maybe i dont because i put  
more than 30, less than 60 minutes
in on this ), database connection have to be explicitly closed via  
Persistent db,
standard close calls etc wouldnt close the connection whereas
storm does close the connections which then have to be reopened.

Its the opening and closing the connection that I want to avoid,
we want to keep the connection open at all times. We are porting code  
from
perl that uses Apache::DBI and without this functionality, we either  
take a big
performance hit or we have to restructure all the code.

Obviously, restructuring everything is something I want to avoid.

Suggestions on getting Apache::DBI persistent connections while using  
Storm?




More information about the storm mailing list