PyPy and bzr

[Wouter van Heyst]

(reordered replies)

> On Sat, May 14, 2011 at 5:33 PM, vila <v.ladeuil+lp at free.fr> wrote:
> > Hey ! Hi Larstiq ! Nice to see you back :)
> >
> > Thanks for working on this.

Heya vila, nice to see you too :)  I'm not quite back yet, but I take a
break from writing my thesis from time to time.

> > <snip/>
> >
> >    > Before I fall asleep, I put up https://code.launchpad.net/~larstiq/bzr/bzr-pypy
> >    > which makes the export and test_pack blackbox tests pass on pypy.
> >
> > I turned it into a merge proposal:
> > https://code.launchpad.net/~larstiq/bzr/bzr-pypy/+merge/60986

Ah thanks, I need to learn some new launchpad functionality :)


On Sat, May 14, 2011 at 18:22:39 +0900, INADA Naoki wrote:
> Today I update lp:bzr checkout and selftest fails with toomanyopenfiles.
> I can see there are many deleted .pack files in /proc/PID/fd
> They are all under temporary folder in blackbox test. But it may because just
> order of running tests.

Starting from bzrlib.tests.blackbox.test_diff.TestCheckoutDiff.test_diff_prefix
and sprinkling pdb.set_trace() through the code, I think it's more than that.

(oh, and watching the output of "watch -n 1 ls -lU /proc/`pidof
pypy`/fd", thanks for reminding me!)


After each commit in DiffBase.make_example_branch we leak 3 open files.
I put in enter and exit debug prints in bzrlib.transport.Transport._seek_and_read(),
and get the following output for 1 commit:

    _seek_and_read enter: <open file u'/tmp/testbzr-XPkTi_.tmp/bzrlib.tests.blackbox.test_diff.TestCheckoutDiff.test_diff_prefix/work/.bzr/repository/upload/mg36a7fhyny2vqhatvwj.pack', mode 'rb' at 0x0b869d68>
    _seek_and_read enter: <open file u'/tmp/testbzr-XPkTi_.tmp/bzrlib.tests.blackbox.test_diff.TestCheckoutDiff.test_diff_prefix/work/.bzr/repository/upload/mg36a7fhyny2vqhatvwj.pack', mode 'rb' at 0x0b869e48>
    _seek_and_read enter: <open file u'/tmp/testbzr-XPkTi_.tmp/bzrlib.tests.blackbox.test_diff.TestCheckoutDiff.test_diff_prefix/work/.bzr/repository/upload/mg36a7fhyny2vqhatvwj.pack', mode 'rb' at 0x0b869f28>
    _seek_and_read enter: <open file u'/tmp/testbzr-XPkTi_.tmp/bzrlib.tests.blackbox.test_diff.TestCheckoutDiff.test_diff_prefix/work/.bzr/repository/indices/5e1072a8ea98b1bd3872794ce51ae286.rix', mode 'rb' at 0x0b89d080>
    _seek_and_read exit: <open file u'/tmp/testbzr-XPkTi_.tmp/bzrlib.tests.blackbox.test_diff.TestCheckoutDiff.test_diff_prefix/work/.bzr/repository/indices/5e1072a8ea98b1bd3872794ce51ae286.rix', mode 'rb' at 0x0b89d080>


_seek_and_read yields its data and closes the file in case of an
exception or after we are done yielding everything, so my suspicion is
that the generator isn't fully consumed.


Instead of digging deeper into pack/groupcompress code I'll get back to
thesis writing for now.

HTH,
Wouter van Heyst