lp:bzr/2.0 upgrade --2a runs out of memory

John Arbash Meinel john at arbash-meinel.com
Fri Sep 4 22:32:39 BST 2009


-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Brian de Alwis wrote:
> On 4-Sep-2009, at 3:18 PM, John Arbash Meinel wrote:
>> That would probably be good to see, at least. I'm currently trying to
>> investigate some of the memory consumption issues. I'm guessing we have
>> some sort of dangling reference to something that we thought we had
>> released.
> 
> I've put up the repository at:
> 
>     http://www.cs.usask.ca/~bsd178/tmp/pkgsrc-21
> 
> It's 129MB.  I'm happy to do more testing.
> 
> Brian.
> 

I grabbed a copy, and confirm it is 129MB locally. It seems to have 21
revisions, but 114,000 texts.

The breakdown looks like:
Commits: 21
                      Raw    %    Compressed    %  Objects
Revisions:         10 KiB   0%         7 KiB   0%       21
Inventories:   338463 KiB  44%      4598 KiB   3%       21
Texts:         422344 KiB  55%    123545 KiB  96%   114604
Signatures:         0 KiB   0%         0 KiB   0%        0
Total:         760819 KiB 100%    128152 KiB 100%   114646


As you know, you have a *lot* of files in there:
12.630  Resizing the inventory entry cache from 10240 to 119264

(I think it resizes to ~1.5x the size of an inventory, so you have
around 80k files in your tree)

With only 21 revs, there won't be a lot of progress indication. (It
gives an update every 100 revs.)

At 5 minutes in, I'm at 430MB using bzr.dev on Linux.

Note that the raw size of all of your content is only 760MB, so I really
don't see where we would get to 3.5GB of content. I suppose I could see
double or so based on the code that is doing delta compression, etc.

But 3.5GB certainly sounds like something else is coming into play. (A
memory leak, something strange with our allocation patterns on Mac, etc.)

12min in, the peak memory is 800MB, the current memory is 544MB.

I'm done for the day, but I'll let it run and see what we get.

John
=:->


-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (Cygwin)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAkqhh3cACgkQJdeBCYSNAAOYeACfYNXz69O5TRkr5jPr0TC1huKy
f2wAniiOfAL8bSl9vIV1RhYSLPkbZg6U
=ZZph
-----END PGP SIGNATURE-----



More information about the bazaar mailing list