Merge eats memory

[John A Meinel]

Michael Ellerman wrote:
> Hi guys,
> 
> I haven't had much time to look into this. Thanks Andrew for the pointer to 
> the ref count script, that's been useful.
> 
> It turns out I was versioning my .git directory in my bzr/git repos, which is 
> unnecessary and obviously going to cause me pain when I merge.
> 
> The memory usage is still ridiculously high though, so I think there's 
> something leaking in there.
> 
> michael at swarm1:~/kernels/bzr1$ du -sh .git/ .bzr/ .
> 200M    .git/
> 321M    .bzr/
> 756M    .
> 
> michael at swarm1:~/kernels/bzr1$ find . -type f | wc -l
> 102243
> michael at swarm1:~/kernels/bzr1$ find .bzr -type f | wc -l
> 51127
> michael at swarm1:~/kernels/bzr1$ find .git -type f | wc -l
> 33751
> 
> 
> michael at swarm1:~/kernels/bzr1$ bzr merge ../bzr2/ ./@1
> 
> Peak memory usage about 970 MB, takes about 79 minutes on a P4 2.8 GHz with 
> 512 MB RAM and 512 MB swap.
> 
> Logs and some graphs here: http://michael.ellerman.id.au/files/bzr
> 
> refs.log is the output (munged) from the countrefs.py running every 10 seconds 
> or so. I ctrl-c'ed that run as I got bored of my box being unusable. The last 
> entry looks like:
> 
> Refs	Class
> ----------------------------------------
> 157370	bzrlib.merge.SourceFile
> 157368	bzrlib.inventory.InventoryEntry
> 95615	bzrlib.changeset.ChangesetEntry
> 95614	bzrlib.changeset.ReplaceContents
> 93164	bzrlib.changeset.ChangeUnixPermissions
> 93164	bzrlib.changeset.FileCreate
> 134	bzrlib.commands.Command
> 48	exceptions.Exception
> 34	random.Random
> 27	threading._Verbose
> 
> 
> I haven't had a chance to look at where we might be holding those objects 
> though. Maybe tonight.
> 
> cheers
> 

Well, if I looked at the refs.log file correctly, it looks like you have
a whole lot of exceptions being generated. They seem to be warnings that
are caught somewhere, but it seems likely that the exceptions are
leaking memory.

We should probably look closely at some of the
bzrlib.inventory.InventoryEntry stuff. But I honestly don't know what
generated your refs.log file, so my understanding might be very wrong.

John
=:->
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 251 bytes
Desc: OpenPGP digital signature
Url : https://lists.ubuntu.com/archives/bazaar/attachments/20050531/c70d20ac/attachment.pgp