loggerhead takes a while to fetch revisions

[Robey Pointer]

(See previous post about what loggerhead is.)

One thing I ran into when writing a web front-end was that the  
process of fetching and processing a revision from a bazaar  
repository can take a while.  I don't think I'm doing that much post- 
processing on a revision (the relevant code is history.py:get_change)  
so I suspect something is going on within bzrlib.

I added a caching system that saves the pickled data structure that I  
create after fetching a revision and its delta (specifically: the  
files that are involved), and it helps IMMENSELY.  With a full cache,  
almost every operation can complete in <= 2 seconds, even annotating  
a file like bzr.dev's NEWS file.  Without it, some operations take 2  
or more *minutes*.

Maybe this is normal behavior, but I thought it was worth  
investigating.  I did an lsprof of the get_cache() call and posted it  
here:

	http://www.lag.net/~robey/code/get_change.html

On my lag.net server, I can't do more than about 25 of these calls  
per minute.  Am I doing something wrong?  Am I exercising a bad part  
of bzrlib?

LarstiQ on IRC (is that Wouter?) recommended I try adding explicit  
locking of the bzrlib branch, but it didn't make a significant  
difference.

Caching pickled revisions is a cop-out -- in a big way -- so I'm  
interested in solving this performance problem if there's anything I  
can do.  I'm basically starting this thread to flush out ideas.

The loggerhead branch is here:
	http://www.lag.net/~robey/code/loggerhead/

robey