[Bug 1782647] Re: py3.7 deadlock in threading

[Corey Bryant]

Swapping out ThreadPoolExecutor with GreenThreadPoolExecutor gets rid of
the deadlock. Joshua Harlow suggests [1]: If you have the ability to
specify which executor your code is using, and you are running under
eventlet I'd give preference to the green thread pool executor under
that situation (and if not running under eventlet then prefer the
threadpool executor variant).

[1] http://lists.openstack.org/pipermail/openstack-
dev/2018-July/132473.html

-- 
You received this bug notification because you are a member of Ubuntu
Foundations Bugs, which is subscribed to python3.7 in Ubuntu.
https://bugs.launchpad.net/bugs/1782647

Title:
  py3.7 deadlock in threading

Status in Designate:
  New
Status in eventlet:
  Unknown
Status in Python:
  Unknown
Status in python3.7 package in Ubuntu:
  New

Bug description:
  I have a feeling this is an issue with a dependency that designate is
  using, or even with py3.7 itself, and not directly a designate issue.
  I'm hitting something similar in heat and will append the bug with
  details there once I get them narrowed down to a simple recreate.

  Things seem to get hung up in:

    File "/usr/lib/python3.7/threading.py", line 296, in wait
      waiter.acquire()

  and:

    File "/usr/lib/python3.7/threading.py", line 1048, in _wait_for_tstate_lock
      elif lock.acquire(block, timeout):

  To reproduce with designate, update tox.ini with the following:

  --- a/tox.ini
  +++ b/tox.ini
  @@ -1,6 +1,6 @@
   [tox]
   minversion = 2.0
  -envlist = py35,py27,flake8
  +envlist = py35,py37,py27,flake8
   skipsdist = True

   [testenv]
  @@ -39,6 +39,12 @@ commands =
     {[testenv]commands}
     stestr run '{posargs}'

  +[testenv:py37]
  +basepython = python3.7
  +commands =
  +  {[testenv]commands}
  +  stestr run 'designate\.tests\.test_workers\.test_processing\.TestProcessingExecutor\.(test_execute_multiple_tasks)'
  +
   [testenv:docs]
   basepython = python3
   deps =

  Notice that while updating tox with a py3.7 target, that also runs a
  single test, test_execute_multiple_tasks. The test will hang and
  require a Control-C to cancel it. The results look like this:
  https://paste.ubuntu.com/p/SwXsCcghjt/

  To recreate with py3.7 you can use Ubuntu Cosmic like so:

  lxc launch ubuntu-daily:cosmic c1
  lxc exec c1 /bin/bash
  root at c1:~# git clone https://github.com/openstack/designate
  root at c1:~# #update tox.ini as shown above
  root at c1:~# tox -e py37

To manage notifications about this bug go to:
https://bugs.launchpad.net/designate/+bug/1782647/+subscriptions