Rev graph caching strategy (was Re: trac-bzr performance)

[Ian Clatworthy]

Martin von Gagern wrote:
> Alexander Belchenko wrote:
>> Martin, recently in IRC I've asked you about caching. This is what I have in my mind. Glad there
>> already is some implementation.
> 
> I remember, and I begin to see the applications...
> This will need some careful planning, but I guess there might be
> someinformation worth caching. Will need to think about what and how.
> And how to notice when caches have become invalid, e.g. by overwriting
> pushes to some branch.

Hi Martin,

Firstly, a huge thank-you for the massive amount of energy you've put
into improving trac-bzr in recent times. Trac remains a very popular
tool and nice bzr integration with it is important to many folk, however
tricky that is behind the scenes.

I think the right strategy wrt caching is to do it in one plugin and for
other plugins to leverage that. Otherwise we end up with caching all
over the place (loggerhead, trac-bzr, qbzr, etc.). That's a problem
because we'll need to re-optimise each plugin as the core improves:
caching can be great but also counter-productive. For example, the
caching in fastimport was essential in the bzr 1.6 days but actually
slowed things down when the new format arrived. I recently made
fastimport much faster and less memory hungry by simply disabling it
recently!

I made a start down this path last year. See
http://doc.bazaar-vcs.org/plugins/en/historycache-plugin.html and
https://launchpad.net/bzr-historycache. It would be good if those of us
interested in caching worked together to improve this so multiple
plugins benefit.

I've just upgraded the trunk to 2a and changed the ownership to be bzr
instead of me. If you want to commit to it's trunk and aren't in bzr,
please apply for membership.

Ian C.