usertest results for index2
John Arbash Meinel
john at arbash-meinel.com
Sun Jul 6 18:22:01 BST 2008
-----BEGIN PGP SIGNED MESSAGE-----
Ian Clatworthy wrote:
| csv files attached comparing common operations across
| 3 deep repos - python, mysql 5.1 and OOo:
| * bzr = rev 3524
| * bzr-btree = rev 3524 + index2 plugin
| In the latter case, bzr init --btree-plain; bzr pull
| was used instead of bzr branch. See usertest rev 118
| for the benchmarking code.
| In summary, log one file is much quicker while most
| other things are much the same, if slightly slower.
| These are fully packed repos though so that's probably
| making the current index code look better than it
| really is. Even so, we might want to dig a little deeper
| on some of these results before making a format with
| btree indexes the default.
| Ian C.
Thanks for running this. If you have a chance, I do have a few ideas...
1) I have a feeling this is mostly because the btree code isn't caching
as aggresively as the current graph index code.
2) Run with 'bzr -Dmemory'. With the latest from bzr.dev this will dump
the status information just before the process exits. So you can find
the total memory consumption, etc.
3) There are a couple interesting lines:
By default we don't cache any of the individual lines, but you can
enable it in 'btree_index.py'
# Default max size is 100,000 leave values
self._leaf_value_cache = None # lru_cache.LRUCache(100*1000)
You can set it to an LRU, or make it a plain python dict if you want to
simulate what the current graph indexes are doing.
I would be curious to see the difference on both memory consumption, and
4) This format is really tuned for more "real world" cases where you
don't have a single optimal pack, but you have several. For example, you
could try doing "for i in `seq 8`; bzr commit --unchanged;" just to
shove a few extra packs into the repository. I would like to *think*
that btree will scale better than graph under those conditions. (It is
also something I'm trying to look closely at with pybloom code.)
I realize you are fairly out of commission to do this testing in the
next few days, Robert is at Guadec, and I'm working on merge.... :(
We'll get there, I guess.
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (Cygwin)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org
-----END PGP SIGNATURE-----
More information about the bazaar