[Maas-devel] MongoDB in the test suite

[John Arbash Meinel]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Just some notes about how we would likely integrate Mongo into a test
suite.

1) 'mongod --dbpath=/path/to/somewhere --port=9999 --smallfiles
- --noprealloc --nojournal'

  a) --dbpath lets us put the files wherever we want in the filesystem

  b) --port is pretty obvious if we are going to be starting a server,
     we need to know where to talk to it. We can use the
    'socket.socket()' bind-but-don't-listen trick to get a new-random
    port that we know is not already in use.

  c) 'use admin; db.shutdownServer()' can be used to stop a server you
     are connected to (if you have rights to the admin db). Though if
     we start the server, we can just kill it with SIGINT as well.

  d) --noprealloc
     Mongo defaults to pre-allocating nulled-content on disk (to avoid
     fragmentation.) It pre-allocates by doubling the current size. So
     dbname.0 is 16MB, dbname.1 is 32MB, etc.
     There is also a .ns file that defaults to 16MB. Prealloc seems to
     be 'when the current file is close to full, allocate another one
     to work from in the future'.
     Without this, every db is 64MB (16MB .0, 32MB .1, 16MB .ns), with
     it, we are at least down to 32MB and we might set --nssize to make
     it smaller.

  e) --smallfiles, in my testing this no longer does anything, but it
     probably did in other versions of mongo. From what I can tell in
     the docs, the default size was .0=64MB, whereas on Precise
     dbname.0 is 16MB. We may want to set this just so we can expect it
     to be 16MB.

  f) Passing an invalid path to 'mongod --path' will startup and die in
     30ms. So if we had a clear separation of what are integration
     tests (touch mongo) and what are not, then we could start a mongod
     for each test.
     However, it probably makes more sense to start a mongod for the
     whole test suite, and just use temp-named databases for each test.
     Still not perfect, as each test case then uses at least 32MB. So
     we might use 'collections' as the namespace.
     In console terms the alternatives are:
       > use testabcd
       > db.hardware.save({"content": 1})
     Vs
       > use tests
       > db.testabcd.hardware.save({"content": 1})
     The former is a bit more obvious, the later will save a lot of
     space during the test suite run.
     Note that you can do both:
       > use tests
       > db.testabcd.drop()
     To drop a single collection from a database. Or
       > use testabcd
       > db.dropDatabase()
     To drop the whole database (and free up the space on disk).

     Also, 'time mongo DB donothing.js' takes 20ms. That may be client
     startup time, or connection time, or something. So we might also
     want to maintain a persistent db connection just because of that
     overhead.

     Changing that to 'time mongo DB create_and_drop.js' changes it to
     32ms. So on my hardware, I can create and drop a database in about
     12ms. Note that without '--noprealloc' allocating the extra 32MB
     increases this overhead to 55ms total or 35ms per db.

     Creating namespaces and data is cheap. Dropping them is a bit more
     expensive. Creating 100 collections with 1 item in each and then
     dropping the db costs 67ms. Creating 100 collections then dropping
     them individually and then the db is 147ms.

     So it would be good to use a separate collection per test case,
     dropping the test suite overhead to around 1.5ms, it may not be
     feasible to pass around the collection prefix. In which case,
     we're talking about 12ms overhead if we do a per-test-case db, or
     30-40ms if we were talking about a per-test-case mongod server.

  g) --nojournal is the default on 32-bit, but not on 64-bit. If you
     have journaling, mongo preallocates 256MB*3 files (0.75GB). They
     apparently will never grow beyond 1GB, but that is still
     ridiculous for testing.

I'll post more as I uncover more details. It does look like
integrating mongod in a test infrastructure is doable. Still something
that you probably don't want to have at a 'unittest' level of detail,
but reasonable at a 'integration' test level.

John
=:->
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (Cygwin)
Comment: Using GnuPG with Mozilla - http://www.enigmail.net/

iEYEARECAAYFAlBPJNsACgkQJdeBCYSNAANzCQCbBp4TM0szhBtxSLe/cA20/0zO
AO8AoJ4FYrZFspQcHqxZkg1ZJRjeA3b9
=GJes
-----END PGP SIGNATURE-----