[MERGE][#230567] Faster (local) branch

[Ian Clatworthy]

Aaron Bentley wrote:
> Ian Clatworthy wrote:
>> John Arbash Meinel wrote:

>>> I can say that if it is costing us a lot, then we might want to
>>> reconsider how
>>> we do it.
>> On large trees like Mozilla and OOo, it certainly is costing a lot.
>> The bigger the tree, the higher the cost.
> 
> Could you please explain *what* is costing us a lot?  Your patch is
> fairly opaque.

Sorry. Our current algorithm - always checking for and managing
any existing files in the working tree even though they can't exist
when running bzr branch - is costing us a lot. For example:

$ time ~/bzr/repo/bzr.faster-branch/bzr branch master xx
Branched 12456 revision(s).

real	0m14.950s
user	0m11.113s
sys	0m2.632s

$ time ~/bzr/repo/bzr.dev/bzr branch master yy
Branched 12456 revision(s).

real	0m23.744s
user	0m17.981s
sys	0m3.348s

I've attached a zip file with the 2 callgrind.out files.
In summary, bzr.dev is taking 35% of its time in tt.apply()
and resolve_conflicts(). In my branch, tt._apply_for_build_tree()
takes 9.3% (and nearly all of that - 8% total - is taken in
applying the inventory delta) and resolve_conflicts() isn't
called at all when creating a branch.

> That TODO is stale.  I and Rob worked on that once, but there was no
> observable performance improvement, so I never merged it.

Fair enough. Right now, we're chmod'ing each file where executable
is not None. I changed this to only set the mode is it was True.
Even with hundreds of executable bits set in the mozilla repo, the
time drops from 5% of a big number (24 secs) to < 1% of a smaller
number (15 secs). I can update the TODO if you like.

>> This isn't an issue. One issue is walking 100K files to find that we
>> only need to rename the top few from limbo across.
> 
> Okay, so how can we fix this in the general case?

My initial patch was using a faster code path if a parameter was
passed to build_tree enabling it. John has suggested that detecting
when that fast path is appropriate is better. I agree - detection
ought to be safer and ensure that the fast path is adopted whenever
it's applicable, not just when it's requested. The trick is ensuring
that the detection check is the right one. I'm hoping you and/or Rob
can help me there.

Any thoughts?

Ian C.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: moz-branch-callgrinds.tar.bz2
Type: application/x-bzip
Size: 58788 bytes
Desc: not available
Url : https://lists.ubuntu.com/archives/bazaar/attachments/20080528/73921d58/attachment-0001.bin