Out of Memory a bridge too far

[Liam Routt]

Hey bzr people,

So I think I'm in a position where I finally have to ditch bazaar for 
our largely Windows-based development. I guess I'm coming here for any 
final suggestions which would allow me to keep using what is 
fundamentally a good tool.


My problem is that a number of times this year we've run up against the 
"out of memory" error. Sometimes it hits a repository first on commits, 
sometimes on updates, but either way it has hit us eventually in almost 
every case. The most recent time (yesterday) comes in the midst of our 
final month before release, and has ground my entire team to a halt - no 
one can check in or out. I have to say, though, that other of our data 
has been trapped in a repository which we are unable to checkout or 
update (or more importantly, revert) for a while now; thankfully that 
hasn't been a critical issue for us - we've been able to protect that 
data in other ways.

We, perhaps obviously, have a mixture of source files and data files. 
Some of our data is xml, but much of it is in the form of binary data, 
including moderate sized Photoshop and Flash files. We have effectively 
given up on storing the largest of our Photoshop files in bazaar on more 
than one occasion during the year (and been lured back to try it a 
number of times).

This weekend I'm coming in to work in order to try to separate out as 
many of the binary files from our critical repository as I can. Given 
how urgent the situation is, I expect that I will end up simply 
abandoning all of the project history and creating a new branch or 
repository with the current files. While this isn't going to greatly 
hamper us, it will be a major disruption.

In looking into the bugs and discussion about the out of memory problems 
which bazaar (and other DVCS) has - and I doubt I've read everything - I 
see suggestions that perhaps using 64-bit python on a Windows machine 
might help. I'm looking in to doing that, but I'm not heartened by the 
very manual nature of the install required, from the documentation on 
the website (most of which would not appear to be targeted at the 
current stable or beta downloads which are available); at this point I'm 
not at all convinced that I can fully identify the right python 
dependencies, especially as some seem to be 32-bit-specific, and I can't 
tell for certain whether the tools I've been using (like TortoiseBZR) 
are included, or I will have to chase them up after I install the rest.

It is further disheartening to realize that these problems I've been 
battling for a large part of the year are not new to Bazaar, they are 
just not important to the development team. Years have elapsed since 
these issues were raised, and all sorts of suggestions which sound as 
though they might at least mitigate the problems appear to have gone 
unimplemented.

I understand that "its not on the current roadmap for Canonical 
sponsored work" (Robert Collins, 06/2010), and that Canonical see bzr as 
a tool for versioning of source files only, but I struggle to see how 
that is conveyed by the website for the application in any clear way.

There are compelling reasons to use bzr. For me, the sane tracking of 
renaming and the treatment of directories as first class objects are 
huge advantages over systems I've used in the past, and the clean way in 
which bound repositories work has made using bazaar easy enough for the 
non-technical people on my team. But encountering severe issues which 
make it impossible to keep our data and code in sync, and have on more 
than one occasion, now, locked data away in repositories which we can no 
longer access even from the machines that checked in the content, leaves 
a bad taste indeed.

Just to underline, while I'm disappointed that bazaar is unable to work 
well with the binary data we have, I'm most annoyed at repeatedly 
getting into situations where we have put data in and are unable to 
access it, without any warning at all. That is an unforgivable situation 
for any application which claims to be doing version control for 
projects, as far as I'm concerned. I believe that at the very least the 
bzr team should (finally) make a priority of ensuring that if it can't 
deal with data it does not allow it to be checked in (which of course 
might ultimately mean that working out how to deal with it is required).


As to my current situation, I'm open to suggestions, if anyone has any. 
But at the moment I'm expecting to limp through to the project 
completion and then find a tool which will handle what is a common 
development situation in a more predictable fashion.


Take care,

Liam Routt
Media Saints