SOLVED (was MemoryError in 2.2.0 in odd case: Can I reduce memory requirements?)

[Doug Lee]

Finally got this server upgraded from Python 2.4 to Python 2.5; fixed
the MemoryErrors.  Surprising, but I'll take it...

On Wed, Jan 12, 2011 at 09:52:44AM -0500, Doug Lee wrote:
I am working on a project managed upstream by Subversion and
containing, along with normal versioned source code files, versioned
binary files, some of which are large.  One in particular is 82
megabytes or so in size.  The project is managed by another company,
so policies on what is versioned are out of my control.

I'm trying to manage local changes for a team using Bazaar.  I share
local commits through a common branch on our Internet server, which
(sheepish grin) happens to run FreeBSD 4.11 - very old, but difficult
to upgrade at this point.  The server runs Python 2.4 and uses the
slow Python code in place of things like CElementTree, in case that
matters (but for this issue, I doubt it does).  I use bzr+ssh protocol
to talk to our server.

The issue:  Repository packs and some checkout, push, etc. operations
crash the server bzr process with MemoryErrors.  I'm not convinced that
the problem only occurs in pack/autopack though, because I don't always
see .bzr.log entries indicating packing first.  The server has at least
two gig of memory.  The problems seem to occur when one of the big binary
files is updated.  Example sequence: update local branch by other means
(e.g., from upstream svn), receive update to large binary file, bzr push
to our server, crash on server with MemoryError, local bzr process hangs
for a long time waiting for answers.

The last error I saw follows and occurred in a string join() call
or perhaps the real_write() call surrounding it.  At this point, a
lightweight checkout from this server is not possible, though if I
populate the local branch by other means, I can successfully switch
to this server.

I'm wondering if there's anything I can do to reduce the memory
requirements here.  I understand that my use case is far from normal
though.

More details:

I checked resource limits:  stacksize seems locked to 64K and datasize
to 512 meg (that's to do with how far a process can extend its data
segment via sbrk(), but I don't know if Python would still use that).
All other limits are unlimited.  I can't seem to extend the datasize
limit past 512 meg even as root, so I assume this is set in stone for
some reason.  I believe if bzr hit a stacksize limit, it would get a
SIGSEGV, which I'm not seeing.

Example .bzr.log session:

Wed 2011-01-12 08:03:22 -0500
1.178  bazaar version: 2.2.0
1.195  bzr arguments: [u'serve', u'--inet', u'--directory=/', u'--allow-writes']
1.406  looking for plugins in /var/bzr/home/.bazaar/plugins
2.224  looking for plugins in /usr/local/lib/python2.4/site-packages/bzrlib/plugins
2.515  encoding stdout as osutils.get_user_encoding() 'US-ASCII'
3.362  failed to load compiled extension: No module named _dirstate_helpers_pyx
3.608  WARNING: using slower ElementTree; consider installing cElementTree and make sure it's on your PYTHONPATH
4.103  failed to load compiled extension: No module named _groupcompress_pyx
184.544  Traceback (most recent call last):
  File "/usr/local/lib/python2.4/site-packages/bzrlib/smart/protocol.py", line 907, in accept_bytes
    _StatefulDecoder.accept_bytes(self, bytes)
  File "/usr/local/lib/python2.4/site-packages/bzrlib/smart/protocol.py", line 402, in accept_bytes
    self.state_accept()
  File "/usr/local/lib/python2.4/site-packages/bzrlib/smart/protocol.py", line 1013, in _state_accept_expecting_message_part
    self.done()
  File "/usr/local/lib/python2.4/site-packages/bzrlib/smart/protocol.py", line 1051, in done
    raise errors.SmartMessageHandlerError(sys.exc_info())
SmartMessageHandlerError: The message handler raised an exception:
Traceback (most recent call last):
  File "/usr/local/lib/python2.4/site-packages/bzrlib/smart/protocol.py", line 1049, in done
    self.message_handler.end_received()
  File "/usr/local/lib/python2.4/site-packages/bzrlib/smart/message.py", line 167, in end_received
    self.responder.send_response(self.request_handler.response)
  File "/usr/local/lib/python2.4/site-packages/bzrlib/smart/protocol.py", line 1211, in send_response
    self._write_prefixed_body(response.body)
  File "/usr/local/lib/python2.4/site-packages/bzrlib/smart/protocol.py", line 1138, in _write_prefixed_body
    self._write_func(bytes)
  File "/usr/local/lib/python2.4/site-packages/bzrlib/smart/protocol.py", line 1095, in _write_func
    self.flush()
  File "/usr/local/lib/python2.4/site-packages/bzrlib/smart/protocol.py", line 1099, in flush
    self._real_write_func(''.join(self._buf))
MemoryError



-- 
Doug Lee                 dgl at dlee.org                http://www.dlee.org
SSB BART Group           doug.lee at ssbbartgroup.com   http://www.ssbbartgroup.com
The very smart may feel they have nothing to learn from anyone;
The very wise will find something to learn from everyone.  (7/14/01)

-- 
Doug Lee                 dgl at dlee.org                http://www.dlee.org
SSB BART Group           doug.lee at ssbbartgroup.com   http://www.ssbbartgroup.com
There is more freedom in knowing how to handle pain than in knowing
how to avoid it.  (4/29/01)