[RFC] Out of memory during BzrDir.find_branches

[Andrew Bennetts]

Aaron Bentley wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
> 
> Andrew Bennetts wrote:
> > Aaron Bentley wrote:
> > A lazy iterator is better regardless of the memory bug, because it starts giving
> > results much faster.
> 
> But again, this is sidestepping a bug; transport operations on the local
> filesystem are much, much slower than they ought to be.
> 
> "find" in my bzr repo completes in 0.240s.  "bzr branches" completes in
> 2.637s.  That's with hot cache for both commands.

For me, it's 0.152s vs. 26.5s (both hot cache).  So "bzr branches" is taking
about 0.5s per branch of bzr.dev for me.  And I don't always run this command
with a hot cache (in fact often not, thanks to memory-hungry web browsers...).

Even if it is sidestepping a bug, a lazy iterator still seems like a good idea
to me.  It won't slow things down significantly, and it will start returning
results faster, especially for huge directory trees.

I'm all for fixing whatever bug(s) make find_branches slow.  I don't think that
should prevent us from allowing BzrDir implementations of find_branches to
return a lazy iterator.  It's a simple change with clear benefits today.  If it
was just to sidestep a bug I'd understand your objection.  But it seems to me to
be a good idea independent of the fact that it sidesteps the bug a little (it
doesn't make it faster, it just gives feedback to the user sooner).

-Andrew.