[CHK/MERGE] CHKMap.iter_interesting_nodes() and fetch updates

[John Arbash Meinel]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Robert Collins wrote:
> On Thu, 2008-11-20 at 14:36 -0600, John Arbash Meinel wrote:
>> -----BEGIN PGP SIGNED MESSAGE-----
>> Hash: SHA1
>>
>> This is a small update on the earlier patch. It changes the logic to
>> significantly reduce buffering for interesting nodes. And in the "create
>> a new branch" case, it should not need to buffer any nodes. There are
>> still edge-cases that would cause more buffering than desired, but the
>> "common" cases should be more well behaved.
> 
> I've merged and pushed all this to the brisbane-core branch; Please
> don't block on me on such things - I would have been extremely happy for
> this to have been merged while I was on leave, as long as one other core
> dev had reviewed it.
> 
> Cheers,
> Rob
> 

I don't think it was blocked on you specifically. It was blocked on "one
other core dev had reviewed it". I think Martin and Andrew were both
rather busy with stuff for the 1.10 release. Thanks for looking at it
and merging it.

By the way, are you working on this at all? I'd like to make sure we
aren't bumping into each other on this.

My plan is to work on getting map() and unmap() to be stable, and then
to work on making it possible to use a hash. With a wide-fan-out hash we
can easily do something like 1M keys in a depth 3 tree (root + 1
internal + leaves). And I think a lot of operations are going to be more
critical by depth.

I was also thinking that for the (parent_id,basename) => file_id map we
could actually hash the parent_id to be the key. That would also give us
good fan out and still retain our locality per basename.

Oh, and another very helpful thing there would be to pull out the common
prefix in leaf (and internal) nodes. (path_id, basename) has a lot of
redundancy. I think I calculated you could fit 2x the number of nodes
per leaf if you just pulled out the common prefix.

John
=:->
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (Cygwin)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iEUEARECAAYFAkk0LLsACgkQJdeBCYSNAAO7ywCgjy1oADrOWK1IVbyaBiq8me64
9xUAl2+TSpXSdA9hyuhNdcQRubY5kUo=
=blYO
-----END PGP SIGNATURE-----