[RFC/PLEASETEST] Per-file graph heads detection during commit for pack repositories.

John Arbash Meinel john at arbash-meinel.com
Wed Nov 14 14:31:43 GMT 2007


-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Robert Collins wrote:
> This is a call for testing.
> 
> Those of you dogfooding packs (hopefully every developer at this point!)
> may have noticed that pack commits are slow *when there is a merge*.
> 
> The root cause of this is debatable; some might say its because we
> calculate merge graph details at commit; others might point at
> inefficiencies in the index layer (it uses generators heavily, and needs
> to be converted to use lists more) or friction in the api stack (we have
> to pack/unpack quite a bit still during commit).
> 
> I think we can make it really fast; partly I want to fix the index
> layer, and graph.heads() still makes way to many calls into the index
> layer which means investment there has a high multiplier for
> performance.
> 
> Another thing to try though, is to use the per-file graph again, this is
> hopefully less graph distance to process, at a tradeoff of reading the
> per-file graph data again (which could actually be less data overall in
> some situations).
> 
> So the attached patch (not complete - theres a new public api without
> enough tests) - speeds up commit a bit in this case by going for the
> per-file graph approach.
> 
> It didn't speed it up nearly as much as I had hoped; I haven't yet
> profiled this in detail; I'll look at that some more a bit later,
> hopefully after feedback from other people - does this help or hinder
> commit of merges on packs for you?
> 
> -Rob
> 

+        search_keys = set(revision_ids)
+        if NULL_REVISION in search_keys:
+            search_keys.remove(NULL_REVISION)

Since search_keys is a set, you can use:

  search_keys.discard(NULL_REVISION)

Which removes it if present, and nothing if not.

+            parents = refs[0]
+            if not parents:
+                parents = (NULL_REVISION,)
+            else:
+                parents = refs[0]

You don't need the 'else' clause.

Also, we still don't have the code that searches small packs before large ones.
 Which will probably help when searching for newer revisions.

I guess I should also just profile some merge commits, and see where I see the
problems.

John
=:->

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.7 (Darwin)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFHOwbOJdeBCYSNAAMRAjjqAJkBne7wEkRzPDUCfpJo5ZTilLsJewCeMjwI
hwWLO0L/p8em8vfYumqme0Q=
=q0vn
-----END PGP SIGNATURE-----



More information about the bazaar mailing list