<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=us-ascii">
</head>
<body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; ">
<font class="Apple-style-span" face="Arial">In order to get content every file for for every revision, I was shell'ing out 'bzr cat' with the right parameters. It is pretty slow you if you fork a process for each file revision, probably because Python runtime
is started and bzr code is loaded for each invocation.</font>
<div><font class="Apple-style-span" face="Arial"><br>
</font></div>
<div><font class="Apple-style-span" face="Arial">So I re-wrote it to use bzrlib API and it is order of magnitude faster. However, it is leaking 2-10MB per revision. I tweaked the code to print heap using heapy. The code and stats are below. Notice that along
with other things, count of bzrlib._static_tuple_c.StaticTuple objects keep increasing.</font></div>
<div><font class="Apple-style-span" face="Arial"><br>
</font></div>
<div><font class="Apple-style-span" face="Arial">I am suspecting I am not using the API correctly (may be the cmd_* classes are not supposed to be used in a loop?). </font></div>
<div><font class="Apple-style-span" face="Arial"><br>
</font></div>
<div><font class="Apple-style-span" face="Arial">I'd appreciate any help or hints.</font></div>
<div><font class="Apple-style-span" face="Arial"><br>
</font></div>
<div><font class="Apple-style-span" face="Arial">Thanks</font></div>
<div><font class="Apple-style-span" face="Arial">Abhay</font></div>
<div><br>
</div>
<div>
<div>from bzrlib.builtins import cmd_cat</div>
<div>from bzrlib.builtins import cmd_log</div>
<div>from bzrlib.revisionspec import RevisionSpec</div>
<div>import StringIO</div>
<div>import os</div>
<div>from guppy import hpy</div>
<div>hp = hpy()</div>
<div><br>
</div>
<div># Get contents of a file for specific revision.</div>
<div>def bzr_cat(repository_url, filename, revision):</div>
<div> print "=cat file"</div>
<div> print hp.heap()</div>
<div> os.chdir(repository_url)</div>
<div> spec = RevisionSpec.from_string(revision)</div>
<div> output = StringIO.StringIO()</div>
<div> cmd = cmd_cat()</div>
<div> cmd.outf = output</div>
<div> cmd.run(filename=filename, revision=[spec], name_from_revision=True)</div>
<div> cmd.cleanup_now()</div>
<div> val = output.getvalue()</div>
<div> output.close()</div>
<div> print hp.heap()</div>
<div> return val</div>
</div>
<div><br>
</div>
<div># Output of heapy when script started</div>
<div><span class="Apple-style-span" style="font-family: 'Lucida Grande'; ">Partition of a set of 111157 objects. Total size = 14541184 bytes.</span><span class="Apple-style-span" style="font-family: 'Lucida Grande'; "><br style="word-wrap: break-word; text-rendering: optimizelegibility; word-break: break-word; ">
</span><span class="Apple-style-span" style="font-family: 'Lucida Grande'; ">Index Count % Size % Cumulative % Kind (class / dict of class)</span><span class="Apple-style-span" style="font-family: 'Lucida Grande'; "><br style="word-wrap: break-word; text-rendering: optimizelegibility; word-break: break-word; ">
</span><span class="Apple-style-span" style="font-family: 'Lucida Grande'; "> 0 50169 45 5017480 35 5017480 35 str</span><span class="Apple-style-span" style="font-family: 'Lucida Grande'; "><br style="word-wrap: break-word; text-rendering: optimizelegibility; word-break: break-word; ">
</span><span class="Apple-style-span" style="font-family: 'Lucida Grande'; "> 1 30786 28 2947480 20 7964960 55 tuple</span><span class="Apple-style-span" style="font-family: 'Lucida Grande'; "><br style="word-wrap: break-word; text-rendering: optimizelegibility; word-break: break-word; ">
</span><span class="Apple-style-span" style="font-family: 'Lucida Grande'; "> 2 8229 7 987480 7 8952440 62 function</span><span class="Apple-style-span" style="font-family: 'Lucida Grande'; "><br style="word-wrap: break-word; text-rendering: optimizelegibility; word-break: break-word; ">
</span><span class="Apple-style-span" style="font-family: 'Lucida Grande'; "> 3 8449 8 946288 7 9898728 68 types.CodeType</span><span class="Apple-style-span" style="font-family: 'Lucida Grande'; "><br style="word-wrap: break-word; text-rendering: optimizelegibility; word-break: break-word; ">
</span><span class="Apple-style-span" style="font-family: 'Lucida Grande'; "> 4 839 1 838768 6 10737496 74 dict of type</span><span class="Apple-style-span" style="font-family: 'Lucida Grande'; "><br style="word-wrap: break-word; text-rendering: optimizelegibility; word-break: break-word; ">
</span><span class="Apple-style-span" style="font-family: 'Lucida Grande'; "> 5 290 0 773152 5 11510648 79 dict of module</span><span class="Apple-style-span" style="font-family: 'Lucida Grande'; "><br style="word-wrap: break-word; text-rendering: optimizelegibility; word-break: break-word; ">
</span><span class="Apple-style-span" style="font-family: 'Lucida Grande'; "> 6 999 1 769392 5 12280040 84 dict (no owner)</span><span class="Apple-style-span" style="font-family: 'Lucida Grande'; "><br style="word-wrap: break-word; text-rendering: optimizelegibility; word-break: break-word; ">
</span><span class="Apple-style-span" style="font-family: 'Lucida Grande'; "> 7 843 1 731648 5 13011688 89 type</span><span class="Apple-style-span" style="font-family: 'Lucida Grande'; "><br style="word-wrap: break-word; text-rendering: optimizelegibility; word-break: break-word; ">
</span><span class="Apple-style-span" style="font-family: 'Lucida Grande'; "> 8 658 1 336416 2 13348104 92 dict of class</span><span class="Apple-style-span" style="font-family: 'Lucida Grande'; "><br style="word-wrap: break-word; text-rendering: optimizelegibility; word-break: break-word; ">
</span><span class="Apple-style-span" style="font-family: 'Lucida Grande'; "> 9 209 0 217360 1 13565464 93 dict of bzrlib.option.Option</span></div>
<div><span class="Apple-style-span" style="font-family: 'Lucida Grande'; "><br>
</span></div>
<div><span class="Apple-style-span" style="font-family: 'Lucida Grande'; ">#Output of heapy after a few thousand calls</span></div>
<div><span class="Apple-style-span" style="font-family: 'Lucida Grande'; ">Partition of a set of 437390 objects. Total size = 334836816 bytes.<br style="word-wrap: break-word; text-rendering: optimizelegibility; word-break: break-word; ">
Index Count % Size % Cumulative % Kind (class / dict of class)<br style="word-wrap: break-word; text-rendering: optimizelegibility; word-break: break-word; ">
0 211711 48 302202312 90 302202312 90 bzrlib._static_tuple_c.StaticTuple<br style="word-wrap: break-word; text-rendering: optimizelegibility; word-break: break-word; ">
1 160872 37 19002840 6 321205152 96 str<br style="word-wrap: break-word; text-rendering: optimizelegibility; word-break: break-word; ">
2 1369 0 4491920 1 325697072 97 dict (no owner)<br style="word-wrap: break-word; text-rendering: optimizelegibility; word-break: break-word; ">
3 31504 7 3026832 1 328723904 98 tuple<br style="word-wrap: break-word; text-rendering: optimizelegibility; word-break: break-word; ">
4 8403 2 1008360 0 329732264 98 function<br style="word-wrap: break-word; text-rendering: optimizelegibility; word-break: break-word; ">
5 8643 2 968016 0 330700280 99 types.CodeType<br style="word-wrap: break-word; text-rendering: optimizelegibility; word-break: break-word; ">
6 866 0 863008 0 331563288 99 dict of type<br style="word-wrap: break-word; text-rendering: optimizelegibility; word-break: break-word; ">
7 296 0 784000 0 332347288 99 dict of module<br style="word-wrap: break-word; text-rendering: optimizelegibility; word-break: break-word; ">
8 871 0 755808 0 333103096 99 type<br style="word-wrap: break-word; text-rendering: optimizelegibility; word-break: break-word; ">
9 662 0 337504 0 333440600 100 dict of class</span></div>
<div><br>
</div>
<div><br>
</div>
</body>
</html>