(fwd) reducing/measuring python memory usage
Robert Collins
robertc at robertcollins.net
Thu Nov 29 00:09:55 GMT 2007
On Wed, 2007-11-28 at 15:51 -0800, Wichmann, Mats D wrote:
> bazaar-bounces at lists.canonical.com wrote:
> > I was thinking of mixing http://wingolog.org/pub/mem_usage.py into
> > bzr as a plugin.
> >
> > -Rob
> > --
> > GPG key available at: <http://www.robertcollins.net/keys.txt>.
>
> It would be interesting to see. I've reported on my performance
> issues and suspect a combination of a lot of memory usage computing
> on branches with a lot of history/files plus the need to access
> lots of files in the metadata (with buffer cache "cold") collectively
> putting too much pressure on the vm subsystem, since the two share
> (compete for) memory. The exact same operations are relatively
> crisp on a different system with lots of ram. I'm not sure there's
> any tool that can tell us about the effects of the second part of
> the combination, but I can grub around a little.
>
> Do we know if other currently interesting systems (git, hg, svn,
> etc.) have to look at as many files during operations like commit?
Have you tried packs yet? They access:
your working files (stat, or stat + read + sha1)
your pack indices (4 files per pack, maintained in logarthimically
increasing sizes)
enough data from your .pack files to generate text parents when
committing, to generate deltas.
-Rob
--
GPG key available at: <http://www.robertcollins.net/keys.txt>.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: This is a digitally signed message part
Url : https://lists.ubuntu.com/archives/bazaar/attachments/20071129/890e0ca7/attachment.pgp
More information about the bazaar
mailing list