Slow inventory extraction from weavefile

[Martin Pool]

On 04/10/05, John A Meinel <john at arbash-meinel.com> wrote:

> > I have also noticed this.  Things like "bzr log FILE" has become
> > _really_ slow.

We have to expect some kind of slowdown for the compression, but it's
too slow at the moment.

When I profiled this I found that on my new Ubuntu Breezy install I
didn't have cElementTree, and  that was slowing it a lot.  It seems to
be spending a lot of time in XML deserialization, which is strange
because that shouldn't have changed much between the two versions. 
Perhaps the profiler's confused by the use of generators.

A few things can be done here:

 - parse the weave once and hold in memory

 - make sure to extract each inventory to an object only once; it
might be happening twice at the moment when calculating the delta

> I've also been pondering what if we only store X number of revisions per
> weavefile, but then have more than 1 weavefile at a time.
> So you could do something like 100 revisions. Or maybe 200 revisions,
> with a 100 revision window (1-200, 100-300, 200-400, etc).

Yes, I think that's a good idea.  For inventories we won't want an
accurate annotate, so having the history split over several files
wouldn't be a problem.

--
Martin