[RFC] Out of memory during BzrDir.find_branches

Aaron Bentley aaron at aaronbentley.com
Mon Apr 21 13:26:27 BST 2008


-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Jonathan Lange wrote:
> Taking a peak at the implementation, I notice that it builds all of
> the open branches into a big list. This seems bound to cause problems.
> Would it be a good thing to make BzrDir.find_branches an iterator?

I don't agree that opening a bunch of branches should be expected to
cause problems.  Branches themselves have very little data; they take up
~48k on ext3.  Their repositories take more, but none of that should be
loaded when a branch is initially opened.

When I run "bzr branches" in my Bazaar repository, memory increases by
27M.  This for 186 branches, so each branch is taking 145k.  That's
several orders of magnitude too big, but still, you'd need more than 20
000 branches to OOM my machine.

I think that switching to an iterator is just sidestepping a bug.

> I can imagine that there are other, deeper memory consumption issues
> at play here. Still, making find_branches an iterable means that
> bzr-removable could use it.

Well, I do want us to have an API that is usable, even if I may disagree
that removing branches from a repository is a good idea.  Are you really
using something like 20 000 branches, though?

Aaron
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFIDIfz0F+nu1YWqI0RAvTOAJ4paHtCIgpXJRwwEFP14ZE8f6sBWQCdFNXn
gvidMhIjE2FuMpuKSMtT2cs=
=/08s
-----END PGP SIGNATURE-----



More information about the bazaar mailing list