[Bug 1051935] Re: Fails with SystemError when too many files are open

[Barry Warsaw]

On Oct 03, 2012, at 11:56 PM, Jason Conti wrote:

>import apt
>
>while True:
>  apt_cache = apt.Cache()
>
>It will eventually consume all of the file descriptors and crash with
>the above SystemError. I was not really successful in reproducing with
>straight apt_pkg, though apt_pkg.PackageRecords(self._cache) seems to be
>the line opening the files. It seems like the issue is that the objects
>are not being garbage collected fast enough. If,
>
>gc.collect()
>
>is run each loop it seems oscillate between 59 and 114 fds, instead of
>increasing rapidly until python runs out. Even better is:
>
>del apt_cache._records
>
>which oscillates between 3 and 59 fds. So a fix might be to add an
>explicit close() method to apt.Cache which deletes at least the _records
>object, and perhaps other objects.

It's generally a bad idea to rely on garbage collection to free up external
resources like file descriptors.  Much better to add an explicit API for
releasing those resources.  So your above example would be better written like
so (if the API were available in pyapt):

    while True:
        apt_cache = apt.Cache()
        apt_cache.close()

Ideally, that would run forever.

Another perhaps useful idea, or at least another way of thinking about it,
would be to implement the context manager protocol, so you could do something
like this:

    while True:
        with apt.Cache() as cache:
            # do something with cache
            pass

and this would also guarantee to free up those file descriptors.

-- 
You received this bug notification because you are a member of Ubuntu
Foundations Bugs, which is subscribed to python-apt in Ubuntu.
https://bugs.launchpad.net/bugs/1051935

Title:
  Fails with SystemError when too many files are open

Status in “python-apt” package in Ubuntu:
  New

Bug description:
  This error occurs each time I try to install a .deb file with Gdebi in
  Ubuntu 12.10, after the deb installation finishes.

  ProblemType: Crash
  DistroRelease: Ubuntu 12.10
  Package: gdebi 0.8.5build1
  ProcVersionSignature: Ubuntu 3.5.0-14.19-generic 3.5.3
  Uname: Linux 3.5.0-14-generic x86_64
  ApportVersion: 2.5.1-0ubuntu7
  Architecture: amd64
  CrashCounter: 1
  Date: Mon Sep 17 15:33:34 2012
  ExecutablePath: /usr/share/gdebi/gdebi-gtk
  InstallationMedia: Ubuntu 12.10 "Quantal Quetzal" - Alpha amd64 (20120914)
  InterpreterPath: /usr/bin/python2.7
  PackageArchitecture: all
  ProcCmdline: /usr/bin/python /usr/bin/gdebi-gtk --non-interactive /home/andrei/Downloads/gnomishdark-theme_201208014-1~webupd8_all.deb
  ProcEnviron:
   PATH=(custom, no user)
   LANG=en_US.UTF-8
   SHELL=/bin/bash
   TERM=unknown
  PythonArgs: ['/usr/bin/gdebi-gtk', '--non-interactive', '/home/andrei/Downloads/gnomishdark-theme_201208014-1~webupd8_all.deb']
  SourcePackage: gdebi
  UpgradeStatus: No upgrade log present (probably fresh install)
  UserGroups:

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/python-apt/+bug/1051935/+subscriptions