SOLVED (was MemoryError in 2.2.0 in odd case: Can I reduce memory requirements?)

[John Arbash Meinel]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 1/12/2011 9:51 AM, Doug Lee wrote:
> Finally got this server upgraded from Python 2.4 to Python 2.5; fixed
> the MemoryErrors.  Surprising, but I'll take it...

There are a few things that I know happened in 2.5. one of the biggest
was that Integer and Floating allocators could return memory back to the
pool. In 2.4 if you allocate 100k integers, you always had a pool for
100k integers even if you only were using 1 of them from there on out.

So doing:

i = range(10**6)
j = map(float, i)
del i
del j

Would still have 1 million PyInt objects and 1 million PyFloat objects
allocated from the system. Creating a new int would re-use one of these
objects, but the mem was never returned to be used elsewhere.

off-hand, I don't know anything that specifically caused this, but it is
possible.

The other major factor is whether you have compiled extensions.
cElementTree doesn't matter much anymore, but bzrlib._groupcompress_pyx
can make a huge difference on memory consumption.

John
=:->
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (Cygwin)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAk0t0i8ACgkQJdeBCYSNAAPLWACeJpLEm+FQysvGkGXyzh2iRtVa
fcgAoLeFsbWNNV3I1FrZKAvDKg1EHc+O
=bUMS
-----END PGP SIGNATURE-----