[RFC/PLEASETEST] Per-file graph heads detection during commit for pack repositories.

James Henstridge james at jamesh.id.au
Tue Nov 20 05:55:23 GMT 2007


On 13/11/2007, Robert Collins <robertc at robertcollins.net> wrote:
> This is a call for testing.
>
> Those of you dogfooding packs (hopefully every developer at this point!)
> may have noticed that pack commits are slow *when there is a merge*.
>
> The root cause of this is debatable; some might say its because we
> calculate merge graph details at commit; others might point at
> inefficiencies in the index layer (it uses generators heavily, and needs
> to be converted to use lists more) or friction in the api stack (we have
> to pack/unpack quite a bit still during commit).
>
> I think we can make it really fast; partly I want to fix the index
> layer, and graph.heads() still makes way to many calls into the index
> layer which means investment there has a high multiplier for
> performance.
>
> Another thing to try though, is to use the per-file graph again, this is
> hopefully less graph distance to process, at a tradeoff of reading the
> per-file graph data again (which could actually be less data overall in
> some situations).
>
> So the attached patch (not complete - theres a new public api without
> enough tests) - speeds up commit a bit in this case by going for the
> per-file graph approach.
>
> It didn't speed it up nearly as much as I had hoped; I haven't yet
> profiled this in detail; I'll look at that some more a bit later,
> hopefully after feedback from other people - does this help or hinder
> commit of merges on packs for you?

In my tests of packs, I ran into a few particularly slow "commit of
merge" cases.

In one case, the commit took 1.25 hours to complete without this patch
(this is quite a large tree).  After applying the patch, I was able to
do the same operation in about 12 seconds.

So even if there are further possible performance enhancements, this
one seems worthwhile persuing in and of itself.

James.



More information about the bazaar mailing list