[RFC/PLEASETEST] Per-file graph heads detection during commit for pack repositories.
John Arbash Meinel
john at arbash-meinel.com
Wed Nov 14 14:31:43 GMT 2007
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
Robert Collins wrote:
> This is a call for testing.
>
> Those of you dogfooding packs (hopefully every developer at this point!)
> may have noticed that pack commits are slow *when there is a merge*.
>
> The root cause of this is debatable; some might say its because we
> calculate merge graph details at commit; others might point at
> inefficiencies in the index layer (it uses generators heavily, and needs
> to be converted to use lists more) or friction in the api stack (we have
> to pack/unpack quite a bit still during commit).
>
> I think we can make it really fast; partly I want to fix the index
> layer, and graph.heads() still makes way to many calls into the index
> layer which means investment there has a high multiplier for
> performance.
>
> Another thing to try though, is to use the per-file graph again, this is
> hopefully less graph distance to process, at a tradeoff of reading the
> per-file graph data again (which could actually be less data overall in
> some situations).
>
> So the attached patch (not complete - theres a new public api without
> enough tests) - speeds up commit a bit in this case by going for the
> per-file graph approach.
>
> It didn't speed it up nearly as much as I had hoped; I haven't yet
> profiled this in detail; I'll look at that some more a bit later,
> hopefully after feedback from other people - does this help or hinder
> commit of merges on packs for you?
>
> -Rob
>
+ search_keys = set(revision_ids)
+ if NULL_REVISION in search_keys:
+ search_keys.remove(NULL_REVISION)
Since search_keys is a set, you can use:
search_keys.discard(NULL_REVISION)
Which removes it if present, and nothing if not.
+ parents = refs[0]
+ if not parents:
+ parents = (NULL_REVISION,)
+ else:
+ parents = refs[0]
You don't need the 'else' clause.
Also, we still don't have the code that searches small packs before large ones.
Which will probably help when searching for newer revisions.
I guess I should also just profile some merge commits, and see where I see the
problems.
John
=:->
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.7 (Darwin)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org
iD8DBQFHOwbOJdeBCYSNAAMRAjjqAJkBne7wEkRzPDUCfpJo5ZTilLsJewCeMjwI
hwWLO0L/p8em8vfYumqme0Q=
=q0vn
-----END PGP SIGNATURE-----
More information about the bazaar
mailing list