[MERGE] Cache inventory.knit so we don't download it twice

[Michael Ellerman]

On 7/19/06, John Arbash Meinel <john at arbash-meinel.com> wrote:
> Michael Ellerman wrote:
> > I've done a bit of profiling with this patch (including spill to
> > disk), vs bzr.dev and vs. my KnitData.read_records_iter patch. Graph
> > attached.
> >
> > In short it doesn't seem to have much effect on memory usage. What the
> > graph does show, IMHO, is that my read_records_iter() in arbitrary
> > order patch is Good Stuff (TM), and we should work out what changed
> > recently to cause the memory spike at the end of branching.
> >
>
> As a public API, I think it should return in the order requested, if
> only because that is what we have specified so far.

> For api compatibility, though, I think we can create a
> 'read_records_iter_unsorted', which does what Michael suggests, and then
> read_records_iter, can just thunk into that, and return what it can.

Actually the current docstring says records will be _read_ in the
order given, it doesn't say anything about the yield order, although I
guess it's implied that they're the same :) </pedantic>

> The attached diff is against my 'cache-inventory' branch, with spill to
> disk enabled. But if Michael can run it against his large branches and
> it drops the memory consumption, I'm willing to clean it up for use with
> bzr.dev.

I'll run some tests against it today.

cheers