fast delta generation in brisbane-core - advice/direction needed

[Ian Clatworthy]

Robert Collins wrote:
> On Mon, 2009-03-02 at 20:08 +1000, Ian Clatworthy wrote:

> Well, thats something we need to evaluate. Where is the time going. Is
> the fast path for getting inventory deltas being used? What function
> shows the most time? Note that TreeDelta is an old, essentially
> deprecated API - you may well be calculating a bunch of unneeded stuff
> to fully populate those objects vs using inventory._make_delta. Moving
> layers around will reduce churn I'm sure.

Robert/John,

Thanks for your help on IRC re this Robert.

Here's a few things found while digging and some follow-up Qs ...

Most of time goes in chk_map.iter_changes(). log -v calls this
for each revision (with self & basis) and, IIUIC, it loads the required
nodes for self & basis every time. I wonder if we want to cache loaded
nodes so that subsequent calls (where basis is now self) don't need to
load (some of the) nodes from disk again? At a bare minimum, caching
the root nodes would help I think as _ensure_root() is always called
twice per iter_changes() call.

There are 3 variables in chk_map.iter_changes() that smell like a
pending optimisation but they are never updated:

>         self_seen = set()
>         basis_seen = set()
>         excluded_keys = set()

BTW, excluded_keys *is* looked up but never added to. How were these
intended to be updated?

Ian C.