Improvements in handling new checkouts (for large trees)

[John Arbash Meinel]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1


...
> The first change (bug #737234) was to avoid redundant work when
> accessing the whole inventory. We had a cache in place, but large trees
> could overflow the cache. With the change, walking all entries in a 70k
> tree drops from about >4min down to under 11s.
> 

Turns out this effects more than I thought. 'bzr merge' also triggers
the iter_entries_by_dir behavior.

Under linux:
5729	5779
 3m05s	 30s	time bzr merge ...

I also dug into bzr merge and found:
  https://bugs.launchpad.net/bzr/+bug/759091

Basically, 'bzr merge' is falling back on our generic 'what has changed'
code path, which requires loading the whole state of each revision we
are comparing. 2a format repositories can answer this question much more
efficiently. An example is:

 3s time bzr st -r OLD_TIP..NEW_TIP

Which means we should be able to get that 30s down around 3s. Probably
more like 5-10s once you figure it has to create new files, etc. But
significantly lower than 30s.

> Under linux, the times all look a lot better, because creating a new
> file on disk isn't nearly as expensive. However, that makes the CPU
> overhead benefit much more apparent.
> 
> 5729	5779
>  3m39s	1m03s	bzr co --lightweight
>    13s	   6s	bzr st
>     1s	   1s	bzr st
> 
> John
> =:->

John
=:->
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.10 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAk2kqCgACgkQJdeBCYSNAAMGxACfYxy/MsHkb+g9QkDv0BPHenw2
5bEAn1/A9QEExWq9vdau7kFf3hupZAES
=KyRX
-----END PGP SIGNATURE-----