[storm] Storm thread-safe question

Tue Aug 5 14:21:36 UTC 2014

It looks good.

It will work as far as you don't try to share stores or storm objects
between threads.
On 5 Aug 2014 15:18, "R Pish" <rpishcr at gmail.com> wrote:

> Thank you so much for your detailed explanation sir!
>
> As a matter of fact, we have some helper functions as the "read_user"
> function you posted that need a store to retrieve data. What we are
> currently doing is to have a required parameter in that function which is
> the existing store to use to prevent creating a new store inside the
> function. Something like:
>
> def read_user(store, user_id):
>     user = store.get(User, user_id)
>     return user
>
> from models.meta import zstorm
> class LoginResource:
>     # POST /login
>     def on_post(self, req, resp):
>         store = zstorm.get("store_provider")
>         user = read_user(store, 25)
> ...
>
> So, ideally we would have only 1 store created per request (the one
> created at the request handler of the API).
>
> How does the above look like?
>
> Thanks again,
> Regards
>
>
>
> 2014-08-05 4:20 GMT-06:00 Oscar Campos <oscar.campos at member.fsf.org>:
>
>> Hi.
>>
>> The design that you shown here will work but there is some considerations
>> that maybe you want to have into account.
>>
>> ZStorm doesn’t make your stores non-blocking, it doesn’t works on that
>> way. ZStorm guarantee that if you call it in any thread, it will returns
>> you a valid store for that thread always. This works much better in an
>> scenario where you have a thread pool available.
>>
>> Take into account that you can never used any object that has been
>> returned from a thread in another thread (not even in the min thread) so if
>> you have a function that runs in another execution thread and do something
>> like:
>>
>> def read_user(user_id):
>>     store = zstorm.get(‘store_provider’)
>>     user = store.get(User, user_id)
>>     return user
>>
>> It will not work as Storm lazy evaluate object properties, that means
>> that Storm may or may not hit the database to retrieve the user information
>> in the previous function so the next time that you try to access a property
>> in the object for example:
>>
>> user = read_user(1263)
>> print(user.name)
>>
>> This will raise an exception in the ZStore module as Storm will try to
>> hit the database in order to retrieve the information for the user and is
>> going to try to use the store that is attached to the object itself that
>> has been created in another execution thread and can not be used in any
>> other thread.
>>
>> Another thing to have into consideration is that if you use a database
>> engine that supports transactions like InnoDB or just Postgres (in case
>> that you use Postgres instead of MySQL) you have to make sure to commit
>> your transactions (even for selects) as each store in each thread will have
>> their own idea of what is the database state and will not be aware of
>> changes made from other stores until both stores are committed.
>>
>> Other challenge that you may to archive is serialise access to data that
>> can be suffering of any type of race condition problem, for example, update
>> the cash of a customer. The best way to do so is using some kind of trigger
>> action in the database.
>>
>> Regards.
>>
>> --
>> Join the free software foundation and become free as in freedom
>>
>> On 5 August 2014 at 05:44:03, R Pish (rpishcr at gmail.com) wrote:
>>
>>  So, we started using storm recently and we are still in a development
>> process, but before going any further, we want to know if we are making
>> things right.
>>
>> The application being built is a REST API developed using Falcon
>> Framework.
>>
>> When the application starts, we import a file (models.meta) which has the
>> following code
>>
>> ...
>> database_dsn = "%s://%s:%s@%s:%s/%s" % (db_driver, db_user, db_pwd,
>> db_host, db_port, db_name)
>> from zope.component import provideUtility, getUtility
>> from storm.zope.interfaces import IZStorm
>> from storm.zope.zstorm import global_zstorm
>> provideUtility(global_zstorm, IZStorm)
>> zstorm = getUtility(IZStorm)
>> zstorm.set_default_uri("store_provider", database_dsn)
>>
>> And, in the file that contains the class that handles the calls to a
>> particular entity in the REST API has the following (abbreviated):
>>
>> from models.meta import zstorm
>> class LoginResource:
>>     # POST /login
>>     def on_post(self, req, resp):
>>         store = zstorm.get("store_provider")
>>         user = store.find...
>> ...
>>
>> So, the code above is the basic idea we are currently starting to use.
>> I wanted to know if just by doing: store = zstorm.get("store_provider")
>> in each function that handles a particular request, is enough to guarantee
>> that the app will be thread-safe and, for example, that 2 concurrent
>> requests will get their non-blocking stores and use the database without
>> blocking each other? (storm does this internally and transparently for the
>> developer?)
>>
>>  I read the following in Storm-Manual:
>> "Therefore, the best policy is usually to create a Store object for each
>> thread which needs one. One convenient way to manage this is to implement a
>> "threadsafe" Store manager, which creates a Store for each thread when that
>> thread first requests one, and then keeps the Store cached in a
>> thread-local dictionary. This approach is taken by ZStorm to provide Store
>> objects to each thread."
>>
>>  If our idea above is wrong, what would we could do to make our app
>> thread-safe?
>>
>> Thanks a lot in advance for any help
>> Regards.
>>  --
>> storm mailing list
>> storm at lists.canonical.com
>> Modify settings or unsubscribe at:
>> https://lists.ubuntu.com/mailman/listinfo/storm
>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.ubuntu.com/archives/storm/attachments/20140805/2123f36a/attachment-0001.html>