[RFC] nicer error when unable to commit large files #109115

[Emilis Dambauskas]

Hi,

Could anyone provide some feedback for this fix?
> thanks
>
> Fabio
>

thank you for taking time to fix this :-).


1. I suggest changing error message in Fabio's patch to:

"Failed to commit file %s: not enough memory. Try buying more memory or `bzr
revert %s` and commit again." % (path, path)

See fabio_memory2.patch .


2. I was trying to fix this bug at the EuroPython2008 sprint, but for a
different case (where CommitBuilder.record_entry_contents() calls
self._add_text_to_weave().

See my patch attached -- emilis_memory.patch -- should be applied after
fabio_memory2.patch.

The problem with my patch is that it has no tests written for the fix. I
have doubts if it is possible to write tests for it at all.


3. I heard there are some fixes by Paul Hummer which are not published
anywhere. The fixes should impose a maximum file size for commits. This
seems logical also, because waiting for your memory to fill up could take a
while.


4. If you need to re-create the bug #109114, but are too lazy to wait for GB
sized files to be processed, you can limit memory for the bzr process and
use MB sized files instead. See the bash script testcase_109114.sh
(attached).


5. It is interesting to note that when you add & commit one big file, bzr
needs 1x amount of memory for:
lines = tree.get_file(ie.file_id, path).readlines()
and then ~4x more memory when calling
self._add_text_to_weave(...)
in bzrlib/repository.py class CommitBuilder.

-- 
Emilis Dambauskas

emilis.d at gmail.com
gsm: +370-686-07732
http://emilis.info

-----BEGIN GEEK CODE BLOCK-----
Version: 3.12
GAT/CC/MC/O dpu(-) s:- a- C++ UBLHS++ P(+) L+++ E--- W+++$ N+ o-- K? !w O?
M-@ V? PS+(--) PE Y+>++ PGP t- 5? X+@ R- !tv b+ DI++++ D G e++ h---- r+++
y++++
------END GEEK CODE BLOCK------
-------------- next part --------------
An HTML attachment was scrubbed...
URL: https://lists.ubuntu.com/archives/bazaar/attachments/20080711/8fa2d7f4/attachment-0001.htm 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: fabio_memory2.patch
Type: text/x-diff
Size: 5867 bytes
Desc: not available
Url : https://lists.ubuntu.com/archives/bazaar/attachments/20080711/8fa2d7f4/attachment-0002.bin 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: emilis_memory.patch
Type: text/x-diff
Size: 4536 bytes
Desc: not available
Url : https://lists.ubuntu.com/archives/bazaar/attachments/20080711/8fa2d7f4/attachment-0003.bin 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: testcase_109114.sh
Type: application/x-sh
Size: 726 bytes
Desc: not available
Url : https://lists.ubuntu.com/archives/bazaar/attachments/20080711/8fa2d7f4/attachment-0001.sh