[Bug 1899159] Re: SRU: backport Python 3.9.5 to 20.04 LTS, 20.10 and 21.04

Tue May 25 08:19:42 UTC 2021

all autopkg tests triggered by python3.9 pass, all autopkg tests
triggered by python3-stdlib-extensions pass. except for

1 test failed:
    test_ttk_textonly

this test is already ignored for python3.9, and also will be ignored
with the pending update of 3.8 to 3.8.10, tracked in LP: #1928057.
Suggesting to ignore that one here, the Tcl/Tk bindings are not much in
use, and checked that idle continues to work with the update.

** Tags removed: verification-needed verification-needed-focal verification-needed-groovy verification-needed-hirsute
** Tags added: verification-done verification-done-focal verification-done-groovy verification-done-hirsute

** Description changed:

- Backport python 3.8.6 and 3.9.0 to focal.
+ Backport python 3.9.5 to groovy and focal.

  Regression potential: ...

  Validation: Test results show no regressions, and the archive test
  rebuild doesn't show any regressions.

  Acceptance criteria:
-  - 21.04: 3.9 is the default version. check test suite and autopkg test results
-  - 20.04 LTS and 20.10: not used in the archive, just check test suite
+  - 21.04: 3.9 is the default version. check test suite and autopkg test results
+  - 20.04 LTS and 20.10: not used in the archive, just check test suite

  It's a minor upstream update, consisting of:

  Security
  --------

  - bpo-43434: Creating a :class:`sqlite3.Connection` object now also produces
-   a ``sqlite3.connect`` :ref:`auditing event <auditing>`. Previously this
-   event was only produced by :func:`sqlite3.connect` calls. Patch by Erlend
-   E. Aasland.
+   a ``sqlite3.connect`` :ref:`auditing event <auditing>`. Previously this
+   event was only produced by :func:`sqlite3.connect` calls. Patch by Erlend
+   E. Aasland.

  - bpo-43882: The presence of newline or tab characters in parts of a URL
-   could allow some forms of attacks.
+   could allow some forms of attacks.

-   Following the controlling specification for URLs defined by WHATWG
-   :func:`urllib.parse` now removes ASCII newlines and tabs from URLs,
-   preventing such attacks.
+   Following the controlling specification for URLs defined by WHATWG
+   :func:`urllib.parse` now removes ASCII newlines and tabs from URLs,
+   preventing such attacks.

  - bpo-43472: Ensures interpreter-level audit hooks receive the
-   ``cpython.PyInterpreterState_New`` event when called through the
-   ``_xxsubinterpreters`` module.
+   ``cpython.PyInterpreterState_New`` event when called through the
+   ``_xxsubinterpreters`` module.

  - bpo-36384: :mod:`ipaddress` module no longer accepts any leading zeros in
-   IPv4 address strings. Leading zeros are ambiguous and interpreted as octal
-   notation by some libraries. For example the legacy function
-   :func:`socket.inet_aton` treats leading zeros as octal notatation. glibc
-   implementation of modern :func:`~socket.inet_pton` does not accept any
-   leading zeros. For a while the :mod:`ipaddress` module used to accept
-   ambiguous leading zeros.
+   IPv4 address strings. Leading zeros are ambiguous and interpreted as octal
+   notation by some libraries. For example the legacy function
+   :func:`socket.inet_aton` treats leading zeros as octal notatation. glibc
+   implementation of modern :func:`~socket.inet_pton` does not accept any
+   leading zeros. For a while the :mod:`ipaddress` module used to accept
+   ambiguous leading zeros.

  - bpo-43075: Fix Regular Expression Denial of Service (ReDoS) vulnerability
-   in :class:`urllib.request.AbstractBasicAuthHandler`. The ReDoS-vulnerable
-   regex has quadratic worst-case complexity and it allows cause a denial of
-   service when identifying crafted invalid RFCs. This ReDoS issue is on the
-   client side and needs remote attackers to control the HTTP server.
+   in :class:`urllib.request.AbstractBasicAuthHandler`. The ReDoS-vulnerable
+   regex has quadratic worst-case complexity and it allows cause a denial of
+   service when identifying crafted invalid RFCs. This ReDoS issue is on the
+   client side and needs remote attackers to control the HTTP server.

  - bpo-42800: Audit hooks are now fired for frame.f_code, traceback.tb_frame,
-   and generator code/frame attribute access.
+   and generator code/frame attribute access.

  Core and Builtins
  -----------------

  - bpo-43105: Importlib now resolves relative paths when creating module spec
-   objects from file locations.
+   objects from file locations.

  - bpo-42924: Fix ``bytearray`` repetition incorrectly copying data from the
-   start of the buffer, even if the data is offset within the buffer (e.g.
-   after reassigning a slice at the start of the ``bytearray`` to a shorter
-   byte string).
+   start of the buffer, even if the data is offset within the buffer (e.g.
+   after reassigning a slice at the start of the ``bytearray`` to a shorter
+   byte string).

  Library
  -------

  - bpo-43993: Update bundled pip to 21.1.1.

  - bpo-43937: Fixed the :mod:`turtle` module working with non-default root
-   window.
+   window.

  - bpo-43930: Update bundled pip to 21.1 and setuptools to 56.0.0

  - bpo-43920: OpenSSL 3.0.0: :meth:`~ssl.SSLContext.load_verify_locations`
-   now returns a consistent error message when cadata contains no valid
-   certificate.
+   now returns a consistent error message when cadata contains no valid
+   certificate.

  - bpo-43607: :mod:`urllib` can now convert Windows paths with ``\\?\``
-   prefixes into URL paths.
+   prefixes into URL paths.

  - bpo-43284: platform.win32_ver derives the windows version from
-   sys.getwindowsversion().platform_version which in turn derives the version
-   from kernel32.dll (which can be of a different version than Windows
-   itself). Therefore change the platform.win32_ver to determine the version
-   using the platform module's _syscmd_ver private function to return an
-   accurate version.
+   sys.getwindowsversion().platform_version which in turn derives the version
+   from kernel32.dll (which can be of a different version than Windows
+   itself). Therefore change the platform.win32_ver to determine the version
+   using the platform module's _syscmd_ver private function to return an
+   accurate version.

  - bpo-42248: [Enum] ensure exceptions raised in ``_missing__`` are
  released

  - bpo-43799: OpenSSL 3.0.0: define ``OPENSSL_API_COMPAT`` 1.1.1 to suppress
-   deprecation warnings. Python requires OpenSSL 1.1.1 APIs.
+   deprecation warnings. Python requires OpenSSL 1.1.1 APIs.

  - bpo-43794: Add :data:`ssl.OP_IGNORE_UNEXPECTED_EOF` constants (OpenSSL
-   3.0.0)
+   3.0.0)

  - bpo-43789: OpenSSL 3.0.0: Don't call the password callback function a
-   second time when first call has signaled an error condition.
+   second time when first call has signaled an error condition.

  - bpo-43788: The header files for :mod:`ssl` error codes are now OpenSSL
-   version-specific. Exceptions will now show correct reason and library
-   codes. The ``make_ssl_data.py`` script has been rewritten to use OpenSSL's
-   text file with error codes.
+   version-specific. Exceptions will now show correct reason and library
+   codes. The ``make_ssl_data.py`` script has been rewritten to use OpenSSL's
+   text file with error codes.

  - bpo-43655: :mod:`tkinter` dialog windows are now recognized as dialogs by
-   window managers on macOS and X Window.
+   window managers on macOS and X Window.

  - bpo-43534: :func:`turtle.textinput` and :func:`turtle.numinput` create now
-   a transient window working on behalf of the canvas window.
+   a transient window working on behalf of the canvas window.

  - bpo-43522: Fix problem with
-   :attr:`~ssl.SSLContext.hostname_checks_common_name`. OpenSSL does not copy
-   hostflags from *struct SSL_CTX* to *struct SSL*.
+   :attr:`~ssl.SSLContext.hostname_checks_common_name`. OpenSSL does not copy
+   hostflags from *struct SSL_CTX* to *struct SSL*.

  - bpo-42967: Allow :class:`bytes` ``separator`` argument in
-   ``urllib.parse.parse_qs`` and ``urllib.parse.parse_qsl`` when parsing
-   :class:`str` query strings. Previously, this raised a ``TypeError``.
+   ``urllib.parse.parse_qs`` and ``urllib.parse.parse_qsl`` when parsing
+   :class:`str` query strings. Previously, this raised a ``TypeError``.

  - bpo-43176: Fixed processing of a dataclass that inherits from a frozen
-   dataclass with no fields. It is now correctly detected as an error.
+   dataclass with no fields. It is now correctly detected as an error.

  - bpo-41735: Fix thread locks in zlib module may go wrong in rare case.
-   Patch by Ma Lin.
+   Patch by Ma Lin.

  - bpo-36470: Fix dataclasses with ``InitVar``\s and
-   :func:`~dataclasses.replace()`. Patch by Claudiu Popa.
+   :func:`~dataclasses.replace()`. Patch by Claudiu Popa.

  - bpo-32745: Fix a regression in the handling of ctypes'
-   :data:`ctypes.c_wchar_p` type: embedded null characters would cause a
-   :exc:`ValueError` to be raised. Patch by Zackery Spytz.
+   :data:`ctypes.c_wchar_p` type: embedded null characters would cause a
+   :exc:`ValueError` to be raised. Patch by Zackery Spytz.

  Documentation
  -------------

  - bpo-43959: The documentation on the PyContextVar C-API was clarified.

  - bpo-43938: Update dataclasses documentation to express that
-   FrozenInstanceError is derived from AttributeError.
+   FrozenInstanceError is derived from AttributeError.

  - bpo-43755: Update documentation to reflect that unparenthesized lambda
-   expressions can no longer be the expression part in an ``if`` clause in
-   comprehensions and generator expressions since Python 3.9.
+   expressions can no longer be the expression part in an ``if`` clause in
+   comprehensions and generator expressions since Python 3.9.

  - bpo-43739: Fixing the example code in Doc/extending/extending.rst to
-   declare and initialize the pmodule variable to be of the right type.
+   declare and initialize the pmodule variable to be of the right type.

-- 
You received this bug notification because you are a member of Ubuntu
Foundations Bugs, which is subscribed to python3-stdlib-extensions in
Ubuntu.
https://bugs.launchpad.net/bugs/1899159

Title:
  SRU: backport Python 3.9.5 to 20.04 LTS, 20.10 and 21.04

Status in python3-stdlib-extensions package in Ubuntu:
  Invalid
Status in python3.9 package in Ubuntu:
  Invalid
Status in python3-stdlib-extensions source package in Focal:
  Fix Committed
Status in python3.9 source package in Focal:
  Fix Committed
Status in python3-stdlib-extensions source package in Groovy:
  Fix Committed
Status in python3.9 source package in Groovy:
  Fix Committed
Status in python3-stdlib-extensions source package in Hirsute:
  Fix Committed
Status in python3.9 source package in Hirsute:
  Fix Committed

Bug description:
  Backport python 3.9.5 to groovy and focal.

  Regression potential: ...

  Validation: Test results show no regressions, and the archive test
  rebuild doesn't show any regressions.

  Acceptance criteria:
   - 21.04: 3.9 is the default version. check test suite and autopkg test results
   - 20.04 LTS and 20.10: not used in the archive, just check test suite

  It's a minor upstream update, consisting of:

  Security
  --------

  - bpo-43434: Creating a :class:`sqlite3.Connection` object now also produces
    a ``sqlite3.connect`` :ref:`auditing event <auditing>`. Previously this
    event was only produced by :func:`sqlite3.connect` calls. Patch by Erlend
    E. Aasland.

  - bpo-43882: The presence of newline or tab characters in parts of a URL
    could allow some forms of attacks.

    Following the controlling specification for URLs defined by WHATWG
    :func:`urllib.parse` now removes ASCII newlines and tabs from URLs,
    preventing such attacks.

  - bpo-43472: Ensures interpreter-level audit hooks receive the
    ``cpython.PyInterpreterState_New`` event when called through the
    ``_xxsubinterpreters`` module.

  - bpo-36384: :mod:`ipaddress` module no longer accepts any leading zeros in
    IPv4 address strings. Leading zeros are ambiguous and interpreted as octal
    notation by some libraries. For example the legacy function
    :func:`socket.inet_aton` treats leading zeros as octal notatation. glibc
    implementation of modern :func:`~socket.inet_pton` does not accept any
    leading zeros. For a while the :mod:`ipaddress` module used to accept
    ambiguous leading zeros.

  - bpo-43075: Fix Regular Expression Denial of Service (ReDoS) vulnerability
    in :class:`urllib.request.AbstractBasicAuthHandler`. The ReDoS-vulnerable
    regex has quadratic worst-case complexity and it allows cause a denial of
    service when identifying crafted invalid RFCs. This ReDoS issue is on the
    client side and needs remote attackers to control the HTTP server.

  - bpo-42800: Audit hooks are now fired for frame.f_code, traceback.tb_frame,
    and generator code/frame attribute access.

  Core and Builtins
  -----------------

  - bpo-43105: Importlib now resolves relative paths when creating module spec
    objects from file locations.

  - bpo-42924: Fix ``bytearray`` repetition incorrectly copying data from the
    start of the buffer, even if the data is offset within the buffer (e.g.
    after reassigning a slice at the start of the ``bytearray`` to a shorter
    byte string).

  Library
  -------

  - bpo-43993: Update bundled pip to 21.1.1.

  - bpo-43937: Fixed the :mod:`turtle` module working with non-default root
    window.

  - bpo-43930: Update bundled pip to 21.1 and setuptools to 56.0.0

  - bpo-43920: OpenSSL 3.0.0: :meth:`~ssl.SSLContext.load_verify_locations`
    now returns a consistent error message when cadata contains no valid
    certificate.

  - bpo-43607: :mod:`urllib` can now convert Windows paths with ``\\?\``
    prefixes into URL paths.

  - bpo-43284: platform.win32_ver derives the windows version from
    sys.getwindowsversion().platform_version which in turn derives the version
    from kernel32.dll (which can be of a different version than Windows
    itself). Therefore change the platform.win32_ver to determine the version
    using the platform module's _syscmd_ver private function to return an
    accurate version.

  - bpo-42248: [Enum] ensure exceptions raised in ``_missing__`` are
  released

  - bpo-43799: OpenSSL 3.0.0: define ``OPENSSL_API_COMPAT`` 1.1.1 to suppress
    deprecation warnings. Python requires OpenSSL 1.1.1 APIs.

  - bpo-43794: Add :data:`ssl.OP_IGNORE_UNEXPECTED_EOF` constants (OpenSSL
    3.0.0)

  - bpo-43789: OpenSSL 3.0.0: Don't call the password callback function a
    second time when first call has signaled an error condition.

  - bpo-43788: The header files for :mod:`ssl` error codes are now OpenSSL
    version-specific. Exceptions will now show correct reason and library
    codes. The ``make_ssl_data.py`` script has been rewritten to use OpenSSL's
    text file with error codes.

  - bpo-43655: :mod:`tkinter` dialog windows are now recognized as dialogs by
    window managers on macOS and X Window.

  - bpo-43534: :func:`turtle.textinput` and :func:`turtle.numinput` create now
    a transient window working on behalf of the canvas window.

  - bpo-43522: Fix problem with
    :attr:`~ssl.SSLContext.hostname_checks_common_name`. OpenSSL does not copy
    hostflags from *struct SSL_CTX* to *struct SSL*.

  - bpo-42967: Allow :class:`bytes` ``separator`` argument in
    ``urllib.parse.parse_qs`` and ``urllib.parse.parse_qsl`` when parsing
    :class:`str` query strings. Previously, this raised a ``TypeError``.

  - bpo-43176: Fixed processing of a dataclass that inherits from a frozen
    dataclass with no fields. It is now correctly detected as an error.

  - bpo-41735: Fix thread locks in zlib module may go wrong in rare case.
    Patch by Ma Lin.

  - bpo-36470: Fix dataclasses with ``InitVar``\s and
    :func:`~dataclasses.replace()`. Patch by Claudiu Popa.

  - bpo-32745: Fix a regression in the handling of ctypes'
    :data:`ctypes.c_wchar_p` type: embedded null characters would cause a
    :exc:`ValueError` to be raised. Patch by Zackery Spytz.

  Documentation
  -------------

  - bpo-43959: The documentation on the PyContextVar C-API was
  clarified.

  - bpo-43938: Update dataclasses documentation to express that
    FrozenInstanceError is derived from AttributeError.

  - bpo-43755: Update documentation to reflect that unparenthesized lambda
    expressions can no longer be the expression part in an ``if`` clause in
    comprehensions and generator expressions since Python 3.9.

  - bpo-43739: Fixing the example code in Doc/extending/extending.rst to
    declare and initialize the pmodule variable to be of the right type.

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/python3-stdlib-extensions/+bug/1899159/+subscriptions