[storm] storm performance

[James Henstridge]

On 12/02/2008, Carlos Garcia Campos <carlosgc at gnome.org> wrote:
> Hi all,
>
> I've been using storm in some projects for some time now without
> problems, however I have a project that needs to do a lot of insertions
> and I've noticed a huge performance problem. The project is now much
> slower than the old versions that didn't use storm.
>
> So, I've written some small test cases to make sure no other things in
> my project are causing the problem. Here are some results:
>
> $ time ./sqlite.py
>
> real    0m47.519s
> user    0m44.607s
> sys     0m2.120s
>
> $ time ./stormm.py
>
> real    9m37.220s
> user    9m24.123s
> sys     0m10.809s
>
>
> both scripts insert 700000 records in the database. I've done the same
> test with mysql and the results are quite similar.

The two tests should result in pretty much identical SQL, so the
overhead is likely to be related to object creation.  This is likely
to be true of any ORM for this sort of workload.

If the data you are inserting doesn't take advantage of any of Storm's
features (such as inserting related objects and having it take care of
getting the foreign keys right), it might be best to use direct SQL to
do the inserts (via store.execute()).

You can still use the ORM features for other parts of your app where
the pros outweigh the cons.

James.