[Bug 2007554] Re: parameterized test coroutines not running in lunar

[Olivier Gayot]

Marking invalid for python3.11 and python-parameterized. Subiquity's
test suite might have relied on undefined behavior.

https://github.com/canonical/subiquity/pull/1566 was merged.

** Changed in: python3.11 (Ubuntu)
       Status: New => Invalid

** Changed in: python-parameterized (Ubuntu)
       Status: New => Invalid

** Changed in: subiquity
       Status: In Progress => Fix Committed

-- 
You received this bug notification because you are a member of Ubuntu
Foundations Bugs, which is subscribed to python3.11 in Ubuntu.
https://bugs.launchpad.net/bugs/2007554

Title:
  parameterized test coroutines not running in lunar

Status in subiquity:
  Fix Committed
Status in python-parameterized package in Ubuntu:
  Invalid
Status in python3.11 package in Ubuntu:
  Invalid

Bug description:
  I have upgraded my desktop to lunar recently, which pulls Python 3.11.
  The subiquity unit tests look all green, but all tests that use coroutines + parameterized don't actually run the coroutine. The presence of warnings makes it more or less obvious.

  excerpt
  -------

  subiquity/server/controllers/tests/test_filesystem.py: 12 warnings
    /usr/lib/python3.11/unittest/async_case.py:90: RuntimeWarning: coroutine 'TestGuidedV2.test_blank_disk' was never awaited
      if self._callMaybeAsync(method) is not None:
    Enable tracemalloc to get traceback where the object was allocated.
    See https://docs.pytest.org/en/stable/how-to/capture-warnings.html#resource-warnings for more info.

  subiquity/server/controllers/tests/test_filesystem.py::TestGuidedV2::test_blank_disk_00
    /usr/lib/python3.11/unittest/case.py:678: DeprecationWarning: It is deprecated to return a value that is not None from a test case (<bound method TestGuidedV2.test_blank_disk of <subiquity.server.controllers.tests.test_filesystem.TestGuidedV2 testMethod=test_blank_disk_00>>)
      return self.run(*args, **kwds)

  subiquity/server/controllers/tests/test_filesystem.py::TestGuidedV2::test_blank_disk_01
    /usr/lib/python3.11/unittest/case.py:678: DeprecationWarning: It is deprecated to return a value that is not None from a test case (<bound method TestGuidedV2.test_blank_disk of <subiquity.server.controllers.tests.test_filesystem.TestGuidedV2 testMethod=test_blank_disk_01>>)
      return self.run(*args, **kwds)

  subiquity/server/controllers/tests/test_filesystem.py::TestGuidedV2::test_blank_disk_02
    /usr/lib/python3.11/unittest/case.py:678: DeprecationWarning: It is deprecated to return a value that is not None from a test case (<bound method TestGuidedV2.test_blank_disk of <subiquity.server.controllers.tests.test_filesystem.TestGuidedV2 testMethod=test_blank_disk_02>>)
      return self.run(*args, **kwds)
  [...]

  simple reproducer
  -----------------

  import unittest

  from parameterized import parameterized

  class TestParameterizedCoroutine(unittest.IsolatedAsyncioTestCase):

      def test_function_no_parameterized(self):
          self.fail("This test ran properly")

      async def test_coroutine_no_parameterized(self):
          self.fail("This test ran properly")

      @parameterized.expand([(1, 2)])
      def test_function_parameterized(self, x, y):
          self.fail("This test ran properly")

      @parameterized.expand([(1, 2)])
      async def test_coroutine_parameterized(self, x, y):
          self.fail("This test ran properly")

  In Python3.10 and lower, all 4 tests "fail" with "This test ran properly".
  In Python3.11, the last test "succeeds" and produces the following warnings:

  /usr/lib/python3.11/unittest/async_case.py:90: RuntimeWarning: coroutine 'TestParameterizedCoroutine.test_coroutine_parameterized' was never awaited
    if self._callMaybeAsync(method) is not None:
  RuntimeWarning: Enable tracemalloc to get the object allocation traceback
  /usr/lib/python3.11/unittest/case.py:678: DeprecationWarning: It is deprecated to return a value that is not None from a test case (<bound method TestParameterizedCoroutine.test_coroutine_parameterized of <test_parameterized_coroutines.TestParameterizedCoroutine testMethod=test_coroutine_parameterized_0>>)
    return self.run(*args, **kwds)

  Tested with python3 -m unittest and python3 -m nose with the same
  result.

To manage notifications about this bug go to:
https://bugs.launchpad.net/subiquity/+bug/2007554/+subscriptions