[storm] Benchmarking storm

[Grahame Bowland]

Hi all

I've just done some benchmarking of Storm, as I've found it a little
bit slow while iterating over a large number of result rows from a
query. Attached is a Python program "bench.py" which has equivalent
benchmarks for Storm, Django, and two other tests using the sqlite3
module directly.

The test is very simple - 100,000 rows of data in an sqlite3 database
are iterated over. Nothing is done with the results. Aside from Django
and Storm I benchmark two trivial cases; just iterating over rows from
a sqlite3 module cursor, and iterating over those rows and creating an
trivial object from the result.

Here are the results:

$ time ./bench.py test.db bench_raw
./bench.py test.db bench_raw  0.12s user 0.02s system 98% cpu 0.145 total
$ time ./bench.py test.db bench_raw_obj
./bench.py test.db bench_raw_obj  0.24s user 0.02s system 98% cpu 0.263 total
$ time ./bench.py test.db bench_django
./bench.py test.db bench_django  2.04s user 0.11s system 98% cpu 2.189 total
$ time STORM_CEXTENSIONS=0 ./bench.py test.db bench_storm
STORM_CEXTENSIONS=0 ./bench.py test.db bench_storm  13.87s user 0.08s
system 99% cpu 14.000 total
$ time STORM_CEXTENSIONS=1 ./bench.py test.db bench_storm
STORM_CEXTENSIONS=1 ./bench.py test.db bench_storm  11.44s user 0.10s
system 99% cpu 11.640 total

You can see that Storm is quite a bit slower than the competition,
significantly it's about five times slower than Django's ORM in this
test. If you run my benchmark with "-p" some hotshot profiling
information is printed out.

I'm going to have a look at why Storm isn't doing so well here, but I
thought I'd post to the list - maybe someone can spot something silly
in my test. The above numbers were gathered on a MacOS 10.5 machine,
with Python 2.5.1, Storm 0.14, Django 1.0.2 and the builtin sqlite3
module.

Have fun
Grahame
-------------- next part --------------
A non-text attachment was scrubbed...
Name: bench.py
Type: application/octet-stream
Size: 2827 bytes
Desc: not available
Url : https://lists.ubuntu.com/archives/storm/attachments/20090303/5712100a/attachment.obj