[storm] Storm thread-safe question

[Oscar Campos]

Hi.

The design that you shown here will work but there is some considerations that maybe you want to have into account.

ZStorm doesn’t make your stores non-blocking, it doesn’t works on that way. ZStorm guarantee that if you call it in any thread, it will returns you a valid store for that thread always. This works much better in an scenario where you have a thread pool available.

Take into account that you can never used any object that has been returned from a thread in another thread (not even in the min thread) so if you have a function that runs in another execution thread and do something like:

def read_user(user_id):
    store = zstorm.get(‘store_provider’)
    user = store.get(User, user_id)
    return user

It will not work as Storm lazy evaluate object properties, that means that Storm may or may not hit the database to retrieve the user information in the previous function so the next time that you try to access a property in the object for example:

user = read_user(1263)
print(user.name)

This will raise an exception in the ZStore module as Storm will try to hit the database in order to retrieve the information for the user and is going to try to use the store that is attached to the object itself that has been created in another execution thread and can not be used in any other thread.

Another thing to have into consideration is that if you use a database engine that supports transactions like InnoDB or just Postgres (in case that you use Postgres instead of MySQL) you have to make sure to commit your transactions (even for selects) as each store in each thread will have their own idea of what is the database state and will not be aware of changes made from other stores until both stores are committed.

Other challenge that you may to archive is serialise access to data that can be suffering of any type of race condition problem, for example, update the cash of a customer. The best way to do so is using some kind of trigger action in the database.

Regards.

-- 
Join the free software foundation and become free as in freedom

On 5 August 2014 at 05:44:03, R Pish (rpishcr at gmail.com) wrote:

So, we started using storm recently and we are still in a development process, but before going any further, we want to know if we are making things right.

The application being built is a REST API developed using Falcon Framework.

When the application starts, we import a file (models.meta) which has the following code

...
database_dsn = "%s://%s:%s@%s:%s/%s" % (db_driver, db_user, db_pwd, db_host, db_port, db_name)
from zope.component import provideUtility, getUtility
from storm.zope.interfaces import IZStorm
from storm.zope.zstorm import global_zstorm
provideUtility(global_zstorm, IZStorm)
zstorm = getUtility(IZStorm)
zstorm.set_default_uri("store_provider", database_dsn)

And, in the file that contains the class that handles the calls to a particular entity in the REST API has the following (abbreviated):

from models.meta import zstorm
class LoginResource:
    # POST /login
    def on_post(self, req, resp):
        store = zstorm.get("store_provider")
        user = store.find...
...

So, the code above is the basic idea we are currently starting to use.
I wanted to know if just by doing: store = zstorm.get("store_provider") in each function that handles a particular request, is enough to guarantee that the app will be thread-safe and, for example, that 2 concurrent requests will get their non-blocking stores and use the database without blocking each other? (storm does this internally and transparently for the developer?)

I read the following in Storm-Manual:
"Therefore, the best policy is usually to create a Store object for each thread which needs one. One convenient way to manage this is to implement a "threadsafe" Store manager, which creates a Store for each thread when that thread first requests one, and then keeps the Store cached in a thread-local dictionary. This approach is taken by ZStorm to provide Store objects to each thread."

If our idea above is wrong, what would we could do to make our app thread-safe?

Thanks a lot in advance for any help
Regards.
--  
storm mailing list  
storm at lists.canonical.com  
Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/storm  
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.ubuntu.com/archives/storm/attachments/20140805/1d452640/attachment.html>