RFC: dirstate-key-cache?

[John Arbash Meinel]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Robert Collins wrote:
> On Tue, 2007-09-25 at 15:50 -0500, John Arbash Meinel wrote:
> 
> 

...

>> I'm not sure what you need to use them for, either. Since we have the
>> optimized cmp_by_dirs and cmp_path_by_dirblock functions. Which can do
>> the comparison without having to .split() at all. (Which is one of the
>> reasons why the C bisect is 17x faster.)
> 
> So, I lost 4 seconds of performance adding the stat cache back in, and
> about 4 of those seconds are in _get_entry. Which is why I was wondering
> about a cache, and the C extensions didn't help at all with the
> performance there.
> 
> -Rob

Out of curiosity, which _get_entry call? And the performance of what?
(commit? status?)

It would be more interesting to see a bit of your lsprof results. There
is quite a few layers for DirState._get_entry() in that it splits the
path, and then goes to _get_block_entry_index, which calls
_find_block_index_from_key, which at least uses the optimized bisect
function.

I did that optimization because I was doing DirState.add() 50k times as
part of the benchmarks, and I saw that it was spending an awful lot
amount of time bisecting. (IIRC it was doing it 2 times, 1 to get the
containing directories entry, and a second to get the location where it
should put the current record.)

Now, if you were going by file_id through _get_entry (rather than by
path), I would expect at least some overhead at building up the file_id
index. (4s for a huge tree might be possible. It seems a bit long, but
not impossibly so.)

So it might be _get_id_index() which is actually hurting you.

I'm just curious why having the stat cache is causing you to call
_get_entry enough times to cause problems. Overall, I feel very unsure
about what you are doing to give any more info here.

John
=:->
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFG+Xr9JdeBCYSNAAMRAgx8AJ9Ce+r9u5wbU+sJDAEA3G9WjEE5rwCdEQPq
B1ZQn+wIPA0s4R1w4wy/56g=
=A62q
-----END PGP SIGNATURE-----