bzr-history-db locking

[John Arbash Meinel]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

bzr-history-db uses an sqlite table as the shared storage. Which does do
concurrent readers and single writer. There can be lock contention, but

 a) When it was discussed as to how it would integrate with launchpad,
    we were going to do 1 shared db per project, not 1 globally shared
    db.

 b) sqlite supports writing concurrent with reading until
    1) you actually finish the commit
    2) the in-memory write buffer fills up

    However, since this is a launchpad installation, we can make the
    write buffer pretty big (figure make it big enough to do a single
    complete history, which should still only be ~100MB)

    At that point, nothing should block the readers, except a short
    interruption for the actual commit this new content.

 c) We also were thinking to pre-populate the larger projects (like
    emacs, etc)
    At that point, the 'writer' updates would then only involve the
    incremental changes vs trunk. Which should always be pretty quick.
    The updating code was definitely written to avoid O(history)
    operations.

 d) Possibly the big blocker is going to be if 2 people push up 2 new
    branches and then access the loggerhead pages for both
    simultaneously, since then the writer would have to check both.

 e) There was an in-process threading.Lock that serialized writer
    threads.


Some thoughts:
 1) Have the branch scanner opportunistically update the bzr-history-db
    for 'active' projects
 2) simple sequential locking for multiple writers using some sort of
    disk lock / polling
    We already have the sqlite lock, that will prevent "INSERT INTO"
    from working, we could just trap that and spin (for a given
    timeout, etc)

John
=:->
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (Cygwin)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAkycsnYACgkQJdeBCYSNAAMZmwCgsKRCD27/o0xkJ9xJeE8rpE+H
pvIAnivPRQR/QbD/1AxI64fzDcxUUeoT
=Kl9z
-----END PGP SIGNATURE-----