[Bug 2075337] Re: py3clean fails when using alternate character set

[Robie Basak]

Investigating this a bit further, /usr/share/python3/debpython/files.py
is calling "dpkg -L <package>" and when that fails to decode as UTF-8 we
get the error.

I tried that for cloud-init and this in stdout for "dpkg -L cloud-init":

/etc/systemd/system/sshd-keygen at .service.d/disable-sshd-keygen-if-cloud-init-active.conf
/lib
détourné par base-files vers : /lib.usr-is-merged
/lib/systemd
/lib/systemd/system

So it seems that the issue only occurs if it lists a diversion (such as
the base-files usrmerge diversion) in the output and that is being
translated out of ASCII 7-bit.

-- 
You received this bug notification because you are a member of Ubuntu
Foundations Bugs, which is subscribed to python3-defaults in Ubuntu.
https://bugs.launchpad.net/bugs/2075337

Title:
  py3clean fails when using alternate character set

Status in python3-defaults package in Ubuntu:
  New

Bug description:
  We noticed this in bug 2060769 but think it is a separate issue.

  dh_python3 is putting a py3clean invocation into package prerm
  maintainer scripts. When that runs, it explodes if the system
  character set is for example ISO-8859-1.

  This appears to affect every package for which dh_python3 has inserted
  a call to py3clean into the package prerm.

  python3-minimal 3.12.3-0ubuntu1

  In this reproducer, cloud-init is at 24.1.3-0ubuntu3.3.

  Steps to reproduce:

  $ lxc launch ubuntu:noble rbasak-pro-l8n
  $ lxc shell rbasak-pro-l8n
  # locale-gen fr_FR
  Generating locales (this might take a while)...
    fr_FR.ISO-8859-1... done
  Generation complete.
  # update-locale LANG=fr_FR
  # exit
  # adjust terminal to ISO-8859-1
  $ lxc shell rbasak-pro-l8n
  # apt update && apt install --reinstall cloud-init

  Expected results: success

  Actual results:

  Lecture des listes de paquets... Fait
  Construction de l'arbre des dépendances... Fait
  Lecture des informations d'état... Fait        
  0 mis à jour, 0 nouvellement installés, 1 réinstallés, 0 à enlever et 5 non mis à jour.
  Il est nécessaire de prendre 598 ko dans les archives.
  Après cette opération, 0 o d'espace disque supplémentaires seront utilisés.
  Réception de :1 http://archive.ubuntu.com/ubuntu noble-updates/main amd64 cloud-init all 24.1.3-0ubuntu3.3 [598 kB]
  598 ko réceptionnés en 1s (926 ko/s)
  Préconfiguration des paquets...
  (Lecture de la base de données... 34406 fichiers et répertoires déjà installés.)
  Préparation du dépaquetage de .../cloud-init_24.1.3-0ubuntu3.3_all.deb ...
  Traceback (most recent call last):
    File "/usr/bin/py3clean", line 210, in <module>
      main()
    File "/usr/bin/py3clean", line 196, in main
      pfiles = set(dpf.from_package(options.package))
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    File "/usr/share/python3/debpython/files.py", line 54, in from_package
      stdout = str(stdout, 'utf-8')
               ^^^^^^^^^^^^^^^^^^^^
  UnicodeDecodeError: 'utf-8' codec can't decode byte 0xe9 in position 2417: invalid continuation byte
  dpkg: avertissement: le sous-processus ancien paquet cloud-init script pre-removal a renvoyé un état de sortie d'erreur 1
  dpkg: tentative d'exécution du script du nouveau paquet à la place...
  Traceback (most recent call last):
    File "/usr/bin/py3clean", line 210, in <module>
      main()
    File "/usr/bin/py3clean", line 196, in main
      pfiles = set(dpf.from_package(options.package))
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    File "/usr/share/python3/debpython/files.py", line 54, in from_package
      stdout = str(stdout, 'utf-8')
               ^^^^^^^^^^^^^^^^^^^^
  UnicodeDecodeError: 'utf-8' codec can't decode byte 0xe9 in position 2417: invalid continuation byte
  dpkg: erreur de traitement de l'archive /var/cache/apt/archives/cloud-init_24.1.3-0ubuntu3.3_all.deb (--unpack) :
   le sous-processus nouveau cloud-init paquet pre-removal script a renvoyé un état de sortie d'erreur 1
  dmesg: read kernel buffer failed: Operation not permitted
                                                           Traceback (most recent call last):
    File "/usr/bin/py3compile", line 323, in <module>
      main()
    File "/usr/bin/py3compile", line 302, in main
      compile(files, versions,
    File "/usr/bin/py3compile", line 185, in compile
      for fn, versions_to_compile in filter_files(files, e_patterns, versions):
    File "/usr/bin/py3compile", line 128, in filter_files
      for fpath in files:
    File "/usr/share/python3/debpython/files.py", line 71, in filter_public
      for fn in files:
    File "/usr/share/python3/debpython/files.py", line 54, in from_package
      stdout = str(stdout, 'utf-8')
               ^^^^^^^^^^^^^^^^^^^^
  UnicodeDecodeError: 'utf-8' codec can't decode byte 0xe9 in position 2417: invalid continuation byte
  dpkg: erreur lors du nettoyage:
   le sous-processus paquet cloud-init script post-installation installé a renvoyé un état de sortie d'erreur 1
  Des erreurs ont été rencontrées pendant l'exécution :
   /var/cache/apt/archives/cloud-init_24.1.3-0ubuntu3.3_all.deb
  E: Sub-process /usr/bin/dpkg returned an error code (1)

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/python3-defaults/+bug/2075337/+subscriptions