usertest results for index2

John Arbash Meinel john at
Sun Jul 6 18:22:01 BST 2008

Hash: SHA1

Ian Clatworthy wrote:
| csv files attached comparing common operations across
| 3 deep repos - python, mysql 5.1 and OOo:
| * bzr = rev 3524
| * bzr-btree = rev 3524 + index2 plugin
| In the latter case, bzr init --btree-plain; bzr pull
| was used instead of bzr branch. See usertest rev 118
| for the benchmarking code.
| In summary, log one file is much quicker while most
| other things are much the same, if slightly slower.
| These are fully packed repos though so that's probably
| making the current index code look better than it
| really is. Even so, we might want to dig a little deeper
| on some of these results before making a format with
| btree indexes the default.
| Ian C.

Thanks for running this. If you have a chance, I do have a few ideas...

1) I have a feeling this is mostly because the btree code isn't caching
as aggresively as the current graph index code.

2) Run with 'bzr -Dmemory'. With the latest from this will dump
the status information just before the process exits. So you can find
the total memory consumption, etc.

3) There are a couple interesting lines:

By default we don't cache any of the individual lines, but you can
enable it in ''

# Default max size is 100,000 leave values
self._leaf_value_cache = None # lru_cache.LRUCache(100*1000)

You can set it to an LRU, or make it a plain python dict if you want to
simulate what the current graph indexes are doing.

I would be curious to see the difference on both memory consumption, and
performance timing.

4) This format is really tuned for more "real world" cases where you
don't have a single optimal pack, but you have several. For example, you
could try doing "for i in `seq 8`; bzr commit --unchanged;" just to
shove a few extra packs into the repository. I would like to *think*
that btree will scale better than graph under those conditions. (It is
also something I'm trying to look closely at with pybloom code.)

I realize you are fairly out of commission to do this testing in the
next few days, Robert is at Guadec, and I'm working on merge.... :(
We'll get there, I guess.

Version: GnuPG v1.4.9 (Cygwin)
Comment: Using GnuPG with Mozilla -


More information about the bazaar mailing list