2GB limit

[Maritza Mendez]

Well that is strange.  The repo in question was freshly imported.
Revno 1.  I note that the client machine which reported the error has
2GB of ram but the server machine has only 512MB and the peak RSS for
the bzr serve process was 500 MB just before the error.

I see that some MemoryError diagnostics landed a few hours ago.
Should be interesting when I get some time to try.

Thanks for the I insight into how to think about bzr memory use.

~M

On 10/12/10, John Arbash Meinel <john at arbash-meinel.com> wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
>
>> Ok.  Thanks for confirming that I have a test case here, based on the
>> groupcompress messages.
>
>> Now here's something.  The branch in question has 1.25 GB distributed
>> (very unevenly) over 700 files.  The largest single file is 195MB and
>> the cumulative size distribution looks like this:
>
>>    Size(MB)    Total Number of Files Larger than Size
>>    =======    ============================
>>    100               2
>>     10             15
>>      1            173
>
>> The second largest file is ~120MB.  So could the ca, 419MB number come
>> from a pack which is combining texts?
>
>> Thanks
>> ~M
>
> I can't say for sure. It is unlikely for us to combine 2 file histories
> into one group if it is larger than 4MB. The current logic should be:
>
> a) If the current group has all the same file, it is allowed to grow to
>    2x the size of the largest content or 4MB, whichever is larger.
> b) If a group has mixed content, it is capped at 2MB
>
> So if you have 10,000 revisions of a 1KB file (with no delta
> compression), it should pack into 3 groups (4MB, 4MB, 2MB).
>
> If you have 10 files that are all 1MB (no delta compression) it will be
> in 5 groups (2MB, ...)
>
> If you have 3 versions of an ISO and get some delta compression, it
> should end up in a very large group (700MB base, + <700MB of deltas).
>
> Now it is possible that the "419" includes the compressed and
> uncompressed sizes. Though that is still only 240MB...
>
> John
> =:->
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1.4.9 (Cygwin)
> Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/
>
> iEYEARECAAYFAky0bOYACgkQJdeBCYSNAAMlaACdG3MNEtruBYhaN/QWcsmthvgU
> cp4AmwQt9L7fX29A05m6TfR45qWKrhwP
> =Hpin
> -----END PGP SIGNATURE-----
>