[storm] Storm thread-safe question

[R Pish]

Excellent. As always, thanks a lot Oscar for your dedication and time to
help us with these questions.
We will keep an eye on that and avoid sharing stores between threads.

Kind regards


2014-08-05 8:21 GMT-06:00 Oscar Campos <oscar.campos at member.fsf.org>:

> It looks good.
>
> It will work as far as you don't try to share stores or storm objects
> between threads.
> On 5 Aug 2014 15:18, "R Pish" <rpishcr at gmail.com> wrote:
>
>> Thank you so much for your detailed explanation sir!
>>
>> As a matter of fact, we have some helper functions as the "read_user"
>> function you posted that need a store to retrieve data. What we are
>> currently doing is to have a required parameter in that function which is
>> the existing store to use to prevent creating a new store inside the
>> function. Something like:
>>
>> def read_user(store, user_id):
>>     user = store.get(User, user_id)
>>     return user
>>
>> from models.meta import zstorm
>> class LoginResource:
>>     # POST /login
>>     def on_post(self, req, resp):
>>         store = zstorm.get("store_provider")
>>         user = read_user(store, 25)
>> ...
>>
>> So, ideally we would have only 1 store created per request (the one
>> created at the request handler of the API).
>>
>> How does the above look like?
>>
>> Thanks again,
>> Regards
>>
>>
>>
>> 2014-08-05 4:20 GMT-06:00 Oscar Campos <oscar.campos at member.fsf.org>:
>>
>>> Hi.
>>>
>>> The design that you shown here will work but there is some
>>> considerations that maybe you want to have into account.
>>>
>>> ZStorm doesn’t make your stores non-blocking, it doesn’t works on that
>>> way. ZStorm guarantee that if you call it in any thread, it will returns
>>> you a valid store for that thread always. This works much better in an
>>> scenario where you have a thread pool available.
>>>
>>> Take into account that you can never used any object that has been
>>> returned from a thread in another thread (not even in the min thread) so if
>>> you have a function that runs in another execution thread and do something
>>> like:
>>>
>>> def read_user(user_id):
>>>     store = zstorm.get(‘store_provider’)
>>>     user = store.get(User, user_id)
>>>     return user
>>>
>>> It will not work as Storm lazy evaluate object properties, that means
>>> that Storm may or may not hit the database to retrieve the user information
>>> in the previous function so the next time that you try to access a property
>>> in the object for example:
>>>
>>> user = read_user(1263)
>>> print(user.name)
>>>
>>> This will raise an exception in the ZStore module as Storm will try to
>>> hit the database in order to retrieve the information for the user and is
>>> going to try to use the store that is attached to the object itself that
>>> has been created in another execution thread and can not be used in any
>>> other thread.
>>>
>>> Another thing to have into consideration is that if you use a database
>>> engine that supports transactions like InnoDB or just Postgres (in case
>>> that you use Postgres instead of MySQL) you have to make sure to commit
>>> your transactions (even for selects) as each store in each thread will have
>>> their own idea of what is the database state and will not be aware of
>>> changes made from other stores until both stores are committed.
>>>
>>> Other challenge that you may to archive is serialise access to data that
>>> can be suffering of any type of race condition problem, for example, update
>>> the cash of a customer. The best way to do so is using some kind of trigger
>>> action in the database.
>>>
>>> Regards.
>>>
>>> --
>>> Join the free software foundation and become free as in freedom
>>>
>>> On 5 August 2014 at 05:44:03, R Pish (rpishcr at gmail.com) wrote:
>>>
>>>  So, we started using storm recently and we are still in a development
>>> process, but before going any further, we want to know if we are making
>>> things right.
>>>
>>> The application being built is a REST API developed using Falcon
>>> Framework.
>>>
>>> When the application starts, we import a file (models.meta) which has
>>> the following code
>>>
>>> ...
>>> database_dsn = "%s://%s:%s@%s:%s/%s" % (db_driver, db_user, db_pwd,
>>> db_host, db_port, db_name)
>>> from zope.component import provideUtility, getUtility
>>> from storm.zope.interfaces import IZStorm
>>> from storm.zope.zstorm import global_zstorm
>>> provideUtility(global_zstorm, IZStorm)
>>> zstorm = getUtility(IZStorm)
>>> zstorm.set_default_uri("store_provider", database_dsn)
>>>
>>> And, in the file that contains the class that handles the calls to a
>>> particular entity in the REST API has the following (abbreviated):
>>>
>>> from models.meta import zstorm
>>> class LoginResource:
>>>     # POST /login
>>>     def on_post(self, req, resp):
>>>         store = zstorm.get("store_provider")
>>>         user = store.find...
>>> ...
>>>
>>> So, the code above is the basic idea we are currently starting to use.
>>> I wanted to know if just by doing: store = zstorm.get("store_provider")
>>> in each function that handles a particular request, is enough to guarantee
>>> that the app will be thread-safe and, for example, that 2 concurrent
>>> requests will get their non-blocking stores and use the database without
>>> blocking each other? (storm does this internally and transparently for the
>>> developer?)
>>>
>>>  I read the following in Storm-Manual:
>>> "Therefore, the best policy is usually to create a Store object for each
>>> thread which needs one. One convenient way to manage this is to implement a
>>> "threadsafe" Store manager, which creates a Store for each thread when that
>>> thread first requests one, and then keeps the Store cached in a
>>> thread-local dictionary. This approach is taken by ZStorm to provide Store
>>> objects to each thread."
>>>
>>>  If our idea above is wrong, what would we could do to make our app
>>> thread-safe?
>>>
>>> Thanks a lot in advance for any help
>>> Regards.
>>>  --
>>> storm mailing list
>>> storm at lists.canonical.com
>>> Modify settings or unsubscribe at:
>>> https://lists.ubuntu.com/mailman/listinfo/storm
>>>
>>>
>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.ubuntu.com/archives/storm/attachments/20140805/01196401/attachment.html>