[Bug 459418] Re: Cache is not safe for concurrent use (by processes or threads)

[Launchpad Bug Tracker]

This bug was fixed in the package lazr.restfulclient - 0.12.0-2ubuntu0.1

---------------
lazr.restfulclient (0.12.0-2ubuntu0.1) quantal-proposed; urgency=low

  * debian/patches/concurrency_fixes_rev_122.patch: backport rev 122: Creates
    AtomicFileCache as a parent class for MultipleRepresentationCache to
    handle concurrent use issues. (LP: #459418)
 -- Mathieu Trudel-Lapierre <mathieu-tl at ubuntu.com>   Fri, 22 Feb 2013 15:01:50 +0000

** Changed in: lazr.restfulclient (Ubuntu Quantal)
       Status: Fix Committed => Fix Released

-- 
You received this bug notification because you are a member of Ubuntu
Foundations Bugs, which is subscribed to lazr.restfulclient in Ubuntu.
https://bugs.launchpad.net/bugs/459418

Title:
  Cache is not safe for concurrent use (by processes or threads)

Status in lazr.restfulclient:
  Fix Released
Status in “lazr.restfulclient” package in Ubuntu:
  Fix Released
Status in “lazr.restfulclient” source package in Precise:
  Fix Released
Status in “lazr.restfulclient” source package in Quantal:
  Fix Released

Bug description:
  [Impact]
  Concurrency issues in lazr.restfulclient are affecting some processes used by various teams; including Foundations, PS integration, etc.
  The most affected users are those who make heavy use of scheduled tasks around launchpadlib (see bug 513116); where different calls to various parts of launchpadlib (or the bzr launchpad plugin) will concurrently try to access cache, step on each others' cache files and mangle the files causing errors for further calls using lazr.restfulclient / launchpadlib.

  [Test case]
  1) Update package.
  2) Run the following command:
  for i in `seq 1 50` ; do python -c 'from launchpadlib.launchpad import Launchpad; lp = Launchpad.login_anonymously("test-cache", "production"); lp.distributions["ubuntu"]' & ; done

  After a few runs, on an affected system the spawned processes will write out backtraces due to mangled files, such as the following error:
  cElementTree.ParseError: not well-formed (invalid token): line 40599, column 18

  On a patched system, all spawned processes will terminate
  successfully.

  [Regression Potential]
  Due to the changes in caching method, it may be possible that lazr.restfulclient fails to write cache files because of path size (which would likely increase due to the change), or otherwise fails to properly cache data increasing delays in retrieving information that may have already been retrieved.

  ----

  httplib2 says:

  class FileCache(object):
      """Uses a local directory as a store for cached files.
      Not really safe to use if multiple threads or processes are going to.
      be running on the same cache.
      """

  launchpadlib sub-classes that, but doesn't change that fact.

  This means that with multiple processes or threads you will
  suddenly get strange error messages (bug 404204).

  This is made worse by the fact that creating an object uses a
  global cache, and login_with does this with no way to override it.

  The best way to fix this is to just fix the cache implementation
  rather than everyone learning to use a private cache and
  fixing launchpadlib to make this possible with the convenience
  methods.

  This is a pre-requisite for fixing bug 513116, which is about the fact
  that launchpadlib is not thread-safe.

To manage notifications about this bug go to:
https://bugs.launchpad.net/lazr.restfulclient/+bug/459418/+subscriptions