[Bug 1899159] Re: SRU: backport Python 3.9.5 to 20.04 LTS, 20.10 and 21.04

[Matthias Klose]

hirsute:

tests look ok, the remaining failures are sshuttle and libreoffice
autopkg test failures, usually ignored, may succeed after several
retries mainly caused by timeouts.

-- 
You received this bug notification because you are a member of Ubuntu
Foundations Bugs, which is subscribed to python3-stdlib-extensions in
Ubuntu.
https://bugs.launchpad.net/bugs/1899159

Title:
  SRU: backport Python 3.9.5 to 20.04 LTS, 20.10 and 21.04

Status in python3-stdlib-extensions package in Ubuntu:
  Invalid
Status in python3.9 package in Ubuntu:
  Invalid
Status in python3-stdlib-extensions source package in Focal:
  Fix Committed
Status in python3.9 source package in Focal:
  Fix Committed
Status in python3-stdlib-extensions source package in Groovy:
  Fix Committed
Status in python3.9 source package in Groovy:
  Fix Committed
Status in python3-stdlib-extensions source package in Hirsute:
  Fix Committed
Status in python3.9 source package in Hirsute:
  Fix Committed

Bug description:
  Backport python 3.8.6 and 3.9.0 to focal.

  Regression potential: ...

  Validation: Test results show no regressions, and the archive test
  rebuild doesn't show any regressions.

  Acceptance criteria:
   - 21.04: 3.9 is the default version. check test suite and autopkg test results
   - 20.04 LTS and 20.10: not used in the archive, just check test suite

  It's a minor upstream update, consisting of:

  Security
  --------

  - bpo-43434: Creating a :class:`sqlite3.Connection` object now also produces
    a ``sqlite3.connect`` :ref:`auditing event <auditing>`. Previously this
    event was only produced by :func:`sqlite3.connect` calls. Patch by Erlend
    E. Aasland.

  - bpo-43882: The presence of newline or tab characters in parts of a URL
    could allow some forms of attacks.

    Following the controlling specification for URLs defined by WHATWG
    :func:`urllib.parse` now removes ASCII newlines and tabs from URLs,
    preventing such attacks.

  - bpo-43472: Ensures interpreter-level audit hooks receive the
    ``cpython.PyInterpreterState_New`` event when called through the
    ``_xxsubinterpreters`` module.

  - bpo-36384: :mod:`ipaddress` module no longer accepts any leading zeros in
    IPv4 address strings. Leading zeros are ambiguous and interpreted as octal
    notation by some libraries. For example the legacy function
    :func:`socket.inet_aton` treats leading zeros as octal notatation. glibc
    implementation of modern :func:`~socket.inet_pton` does not accept any
    leading zeros. For a while the :mod:`ipaddress` module used to accept
    ambiguous leading zeros.

  - bpo-43075: Fix Regular Expression Denial of Service (ReDoS) vulnerability
    in :class:`urllib.request.AbstractBasicAuthHandler`. The ReDoS-vulnerable
    regex has quadratic worst-case complexity and it allows cause a denial of
    service when identifying crafted invalid RFCs. This ReDoS issue is on the
    client side and needs remote attackers to control the HTTP server.

  - bpo-42800: Audit hooks are now fired for frame.f_code, traceback.tb_frame,
    and generator code/frame attribute access.

  Core and Builtins
  -----------------

  - bpo-43105: Importlib now resolves relative paths when creating module spec
    objects from file locations.

  - bpo-42924: Fix ``bytearray`` repetition incorrectly copying data from the
    start of the buffer, even if the data is offset within the buffer (e.g.
    after reassigning a slice at the start of the ``bytearray`` to a shorter
    byte string).

  Library
  -------

  - bpo-43993: Update bundled pip to 21.1.1.

  - bpo-43937: Fixed the :mod:`turtle` module working with non-default root
    window.

  - bpo-43930: Update bundled pip to 21.1 and setuptools to 56.0.0

  - bpo-43920: OpenSSL 3.0.0: :meth:`~ssl.SSLContext.load_verify_locations`
    now returns a consistent error message when cadata contains no valid
    certificate.

  - bpo-43607: :mod:`urllib` can now convert Windows paths with ``\\?\``
    prefixes into URL paths.

  - bpo-43284: platform.win32_ver derives the windows version from
    sys.getwindowsversion().platform_version which in turn derives the version
    from kernel32.dll (which can be of a different version than Windows
    itself). Therefore change the platform.win32_ver to determine the version
    using the platform module's _syscmd_ver private function to return an
    accurate version.

  - bpo-42248: [Enum] ensure exceptions raised in ``_missing__`` are
  released

  - bpo-43799: OpenSSL 3.0.0: define ``OPENSSL_API_COMPAT`` 1.1.1 to suppress
    deprecation warnings. Python requires OpenSSL 1.1.1 APIs.

  - bpo-43794: Add :data:`ssl.OP_IGNORE_UNEXPECTED_EOF` constants (OpenSSL
    3.0.0)

  - bpo-43789: OpenSSL 3.0.0: Don't call the password callback function a
    second time when first call has signaled an error condition.

  - bpo-43788: The header files for :mod:`ssl` error codes are now OpenSSL
    version-specific. Exceptions will now show correct reason and library
    codes. The ``make_ssl_data.py`` script has been rewritten to use OpenSSL's
    text file with error codes.

  - bpo-43655: :mod:`tkinter` dialog windows are now recognized as dialogs by
    window managers on macOS and X Window.

  - bpo-43534: :func:`turtle.textinput` and :func:`turtle.numinput` create now
    a transient window working on behalf of the canvas window.

  - bpo-43522: Fix problem with
    :attr:`~ssl.SSLContext.hostname_checks_common_name`. OpenSSL does not copy
    hostflags from *struct SSL_CTX* to *struct SSL*.

  - bpo-42967: Allow :class:`bytes` ``separator`` argument in
    ``urllib.parse.parse_qs`` and ``urllib.parse.parse_qsl`` when parsing
    :class:`str` query strings. Previously, this raised a ``TypeError``.

  - bpo-43176: Fixed processing of a dataclass that inherits from a frozen
    dataclass with no fields. It is now correctly detected as an error.

  - bpo-41735: Fix thread locks in zlib module may go wrong in rare case.
    Patch by Ma Lin.

  - bpo-36470: Fix dataclasses with ``InitVar``\s and
    :func:`~dataclasses.replace()`. Patch by Claudiu Popa.

  - bpo-32745: Fix a regression in the handling of ctypes'
    :data:`ctypes.c_wchar_p` type: embedded null characters would cause a
    :exc:`ValueError` to be raised. Patch by Zackery Spytz.

  Documentation
  -------------

  - bpo-43959: The documentation on the PyContextVar C-API was
  clarified.

  - bpo-43938: Update dataclasses documentation to express that
    FrozenInstanceError is derived from AttributeError.

  - bpo-43755: Update documentation to reflect that unparenthesized lambda
    expressions can no longer be the expression part in an ``if`` clause in
    comprehensions and generator expressions since Python 3.9.

  - bpo-43739: Fixing the example code in Doc/extending/extending.rst to
    declare and initialize the pmodule variable to be of the right type.

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/python3-stdlib-extensions/+bug/1899159/+subscriptions