CHKMap usage - optimizations?

[Jelmer Vernooij]

On Thu, May 14, 2009 at 12:01:38AM +1000, Robert Collins wrote:
> On Wed, 2009-05-13 at 15:19 +0200, Jelmer Vernooij wrote:
> > Hi Robert,

> > I've got the file id map in bzr-svn based on CHKMap working. Performance
> > is better than with a "plain" Knit as I was using before, but it's still
> > quite slow.

> > I'm storing these file id maps in a batch, in a linear way: basically a
> > lot of iterations of:

> >  * map = CHKMap(store, x)
> >  * analyse log output for a particular revision to determine delta
> >  * x = map.apply_delta(delta)

> > Most of the time is spent in CHKMap at the moment:

> > CHKMap.apply_delta(): 77.63%
> >  * of which: Knit.add_lines(): 47.27%
> > CHKMap.iteritems(): 5.14%

> > What's the best way of improving this performance ? Switching to packs?

> They will be faster.

What's the best place to be looking to get started with this? Can you
give me a few hints?  bzrlib.plugins.search.index is a bit overwhelming..

> > Is there some way I can use transactions here?

> apply_delta is pretty efficient. You are perhaps suffering some
> thrashing on page re-reads, but the basic problem is likely to be kndx
> related.
The 30% that CHKMap.apply_delta() doesn't spend in Knit.add_lines()
seems to be time spent purely in apply_delta itself. 

Cheers,

Jelmer
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 315 bytes
Desc: Digital signature
Url : https://lists.ubuntu.com/archives/bazaar/attachments/20090513/0a169fb0/attachment.pgp