brisbane-core+ric-commit results

[John Arbash Meinel]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1


...

> 
> So for branching Launchpad:
> 
> 44m	Old brisbane-core
> 26m	new brisbane-core
> 17m	set btree leaf cache to 4k
> 
> However, part of this is also because as we are generating the new
> content, we are querying it to ensure that the keys are not duplicated.
> Also, we are spilling the new CHK nodes to disk, and then combining them.
> 
> I tried setting "random_id = True" but I added a check, and for some
> very strange reason, we are sending the same key multiple times. It
> seems to be one of the last steps that we do, so I'm not sure if there
> is a bug in the chk streams (near the end), or if it is a bug with
> signatures, or something else.


So it was a bug in "iter_interesting_nodes()" that was accidentally
sending the same chk page twice. (The bug was if there was a single leaf
node with the same content as a leaf node that was referenced by a
split. In this case the node was the empty tree with only the
TREE_ROOT.) There is a simple fix for it.

So my next test was with btree leaf cache at 1k, but setting
'random_id=True' disabling 'combine_backing_indexes'.

At that point, I was at 14m4s (with lsprof).

So I had some ideas about the btree caching issues, but they don't
really seem to fit. So I'm going to think about them some more. Setting
random_id=True and combine_backing_indices=False had a bigger effect
than btree cache size...

John
=:->
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (Cygwin)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iEYEARECAAYFAknMUbEACgkQJdeBCYSNAAOGgACeM3LziQP0h8ABp5bAYzTWMKhf
wnsAn3fHKvsIev3+2DdbLj3egS46Dv9E
=IkVs
-----END PGP SIGNATURE-----