[storm] Storm thread-safe question

[R Pish]

Thank you so much for your detailed explanation sir!

As a matter of fact, we have some helper functions as the "read_user"
function you posted that need a store to retrieve data. What we are
currently doing is to have a required parameter in that function which is
the existing store to use to prevent creating a new store inside the
function. Something like:

def read_user(store, user_id):
    user = store.get(User, user_id)
    return user

from models.meta import zstorm
class LoginResource:
    # POST /login
    def on_post(self, req, resp):
        store = zstorm.get("store_provider")
        user = read_user(store, 25)
...

So, ideally we would have only 1 store created per request (the one created
at the request handler of the API).

How does the above look like?

Thanks again,
Regards



2014-08-05 4:20 GMT-06:00 Oscar Campos <oscar.campos at member.fsf.org>:

> Hi.
>
> The design that you shown here will work but there is some considerations
> that maybe you want to have into account.
>
> ZStorm doesn’t make your stores non-blocking, it doesn’t works on that
> way. ZStorm guarantee that if you call it in any thread, it will returns
> you a valid store for that thread always. This works much better in an
> scenario where you have a thread pool available.
>
> Take into account that you can never used any object that has been
> returned from a thread in another thread (not even in the min thread) so if
> you have a function that runs in another execution thread and do something
> like:
>
> def read_user(user_id):
>     store = zstorm.get(‘store_provider’)
>     user = store.get(User, user_id)
>     return user
>
> It will not work as Storm lazy evaluate object properties, that means that
> Storm may or may not hit the database to retrieve the user information in
> the previous function so the next time that you try to access a property in
> the object for example:
>
> user = read_user(1263)
> print(user.name)
>
> This will raise an exception in the ZStore module as Storm will try to hit
> the database in order to retrieve the information for the user and is going
> to try to use the store that is attached to the object itself that has been
> created in another execution thread and can not be used in any other thread.
>
> Another thing to have into consideration is that if you use a database
> engine that supports transactions like InnoDB or just Postgres (in case
> that you use Postgres instead of MySQL) you have to make sure to commit
> your transactions (even for selects) as each store in each thread will have
> their own idea of what is the database state and will not be aware of
> changes made from other stores until both stores are committed.
>
> Other challenge that you may to archive is serialise access to data that
> can be suffering of any type of race condition problem, for example, update
> the cash of a customer. The best way to do so is using some kind of trigger
> action in the database.
>
> Regards.
>
> --
> Join the free software foundation and become free as in freedom
>
> On 5 August 2014 at 05:44:03, R Pish (rpishcr at gmail.com) wrote:
>
>  So, we started using storm recently and we are still in a development
> process, but before going any further, we want to know if we are making
> things right.
>
> The application being built is a REST API developed using Falcon Framework.
>
> When the application starts, we import a file (models.meta) which has the
> following code
>
> ...
> database_dsn = "%s://%s:%s@%s:%s/%s" % (db_driver, db_user, db_pwd,
> db_host, db_port, db_name)
> from zope.component import provideUtility, getUtility
> from storm.zope.interfaces import IZStorm
> from storm.zope.zstorm import global_zstorm
> provideUtility(global_zstorm, IZStorm)
> zstorm = getUtility(IZStorm)
> zstorm.set_default_uri("store_provider", database_dsn)
>
> And, in the file that contains the class that handles the calls to a
> particular entity in the REST API has the following (abbreviated):
>
> from models.meta import zstorm
> class LoginResource:
>     # POST /login
>     def on_post(self, req, resp):
>         store = zstorm.get("store_provider")
>         user = store.find...
> ...
>
> So, the code above is the basic idea we are currently starting to use.
> I wanted to know if just by doing: store = zstorm.get("store_provider") in
> each function that handles a particular request, is enough to guarantee
> that the app will be thread-safe and, for example, that 2 concurrent
> requests will get their non-blocking stores and use the database without
> blocking each other? (storm does this internally and transparently for the
> developer?)
>
>  I read the following in Storm-Manual:
> "Therefore, the best policy is usually to create a Store object for each
> thread which needs one. One convenient way to manage this is to implement a
> "threadsafe" Store manager, which creates a Store for each thread when that
> thread first requests one, and then keeps the Store cached in a
> thread-local dictionary. This approach is taken by ZStorm to provide Store
> objects to each thread."
>
>  If our idea above is wrong, what would we could do to make our app
> thread-safe?
>
> Thanks a lot in advance for any help
> Regards.
>  --
> storm mailing list
> storm at lists.canonical.com
> Modify settings or unsubscribe at:
> https://lists.ubuntu.com/mailman/listinfo/storm
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.ubuntu.com/archives/storm/attachments/20140805/8c9f1e17/attachment.html>