[Bug 1986623] Re: cryptsetup fails to decrypt root partion during boot

Sat Nov 26 10:49:22 UTC 2022

Having same issue, just rebooted and now cannot login to the system,
does not accept the password, was working perfectly fine prior to
reboot.

This is on a box that was recently setup with 22.04LTS with ZFS + encryption via the install options. 
Happy to contribute any additional info needed.

-- 
You received this bug notification because you are a member of Ubuntu
Foundations Bugs, which is subscribed to cryptsetup in Ubuntu.
https://bugs.launchpad.net/bugs/1986623

Title:
  cryptsetup fails to decrypt root partion during boot

Status in cryptsetup package in Ubuntu:
  Confirmed
Status in linux package in Ubuntu:
  Incomplete

Bug description:
  During boot, cryptsetup fails to decrypt the root partition in a,
  seemingly, non-deterministic fashion. I know that the password is
  correct and that the keymap is not a fault either, because I have
  specifically chosen the very weak password "123456" for testing
  purposes. A hardware defect seems also rather unlikely as this
  behavior does not affect other Linux distributions or FreeBSD. Earlier
  Ubuntu versions do not seem to be affected either, as this bug appears
  to have been introduced during a kernel update in 20.04 and persists
  throughout 20.04-22.04. Unfortunately I cannot pinpoint the exact
  kernel update that introduced this bug. I have appended the output of
  cryptsetup when manually called from the initramfs shell. Here the
  second attempt succeeded in decrypting the root partition, however, it
  usually takes a lot more attempts to do so.

  As for some additional information, I can decrypt the same luks
  partition from a live USB without any problems whatsoever.

  echo 123456 | cryptsetup open --type luks --debug /dev/nvme0n1p3 nvme0n1p3_crypt
  # cryptsetup 2.4.3 processing "cryptsetup open --type luks --debug /dev/nvme0n1p3 nvme0n1p3_crypt"
  # Running command open.
  # Locking memory.
  # Installing SIGINT/SIGTERM handler.
  # Unblocking interruption on signal.
  # Allocating context for crypt device /dev/nvme0n1p3.
  # Trying to open and read device /dev/nvme0n1p3 with direct-io.
  # Initialising device-mapper backend library.
  # Trying to load any crypt type from device /dev/nvme0n1p3.
  # Crypto backend (OpenSSL 3.0.2 15 Mar 2022 [default]) initialized in cryptsetup library version 2.4.3.
  # Detected kernel Linux 5.15.0-46-generic x86_64.
  # Loading LUKS2 header (repair disabled).
  # Acquiring read lock for device /dev/nvme0n1p3.
  # Opening lock resource file /run/cryptsetup/L_259:3
  # Verifying lock handle for /dev/nvme0n1p3.
  # Device /dev/nvme0n1p3 READ lock taken.
  # Trying to read primary LUKS2 header at offset 0x0.
  # Opening locked device /dev/nvme0n1p3
  # Verifying locked device handle (bdev)
  # LUKS2 header version 2 of size 16384 bytes, checksum sha256.
  # Checksum:99172356e66a2fec247b1e5c758af8bc1338a3fb8bd973aab5e1512a93b2dbdc (on-disk)
  # Checksum:99172356e66a2fec247b1e5c758af8bc1338a3fb8bd973aab5e1512a93b2dbdc (in-memory)
  # Trying to read secondary LUKS2 header at offset 0x4000.
  # Reusing open ro fd on device /dev/nvme0n1p3
  # LUKS2 header version 2 of size 16384 bytes, checksum sha256.
  # Checksum:655a50aef64b6fd4e10b8863df72d7fc62a7020f74684cb3419d4e22adb6fd9c (on-disk)
  # Checksum:655a50aef64b6fd4e10b8863df72d7fc62a7020f74684cb3419d4e22adb6fd9c (in-memory)
  # Device size 497776852992, offset 16777216.
  # Device /dev/nvme0n1p3 READ lock released.
  # PBKDF argon2id, time_ms 2000 (iterations 0), max_memory_kb 1048576, parallel_threads 4.
  # Activating volume nvme0n1p3_crypt using token (any type) -1.
  # dm version   [ opencount flush ]   [16384] (*1)
  # dm versions   [ opencount flush ]   [16384] (*1)
  # Detected dm-ioctl version 4.45.0.
  # Detected dm-crypt version 1.23.0.
  # Device-mapper backend running with UDEV support enabled.
  # dm status nvme0n1p3_crypt  [ opencount noflush ]   [16384] (*1)
  No usable token is available.
  # STDIN descriptor passphrase entry requested.
  # Activating volume nvme0n1p3_crypt [keyslot -1] using passphrase.
  # dm versions   [ opencount flush ]   [16384] (*1)
  # dm status nvme0n1p3_crypt  [ opencount noflush ]   [16384] (*1)
  # Keyslot 0 priority 1 != 2 (required), skipped.
  # Keyslot 1 priority 1 != 2 (required), skipped.
  # Trying to open LUKS2 keyslot 0.
  # Running keyslot key derivation.
  # Reading keyslot area [0x8000].
  # Acquiring read lock for device /dev/nvme0n1p3.
  # Opening lock resource file /run/cryptsetup/L_259:3
  # Verifying lock handle for /dev/nvme0n1p3.
  # Device /dev/nvme0n1p3 READ lock taken.
  # Reusing open ro fd on device /dev/nvme0n1p3
  # Device /dev/nvme0n1p3 READ lock released.
  # Verifying key from keyslot 0, digest 0.
  # Digest 0 (pbkdf2) verify failed with -1.
  # Trying to open LUKS2 keyslot 1.
  # Running keyslot key derivation.
  # Reading keyslot area [0x47000].
  # Acquiring read lock for device /dev/nvme0n1p3.
  # Opening lock resource file /run/cryptsetup/L_259:3
  # Verifying lock handle for /dev/nvme0n1p3.
  # Device /dev/nvme0n1p3 READ lock taken.
  # Reusing open ro fd on device /dev/nvme0n1p3
  # Device /dev/nvme0n1p3 READ lock released.
  # Verifying key from keyslot 1, digest 0.
  # Digest 0 (pbkdf2) verify failed with -1.
  # Releasing crypt device /dev/nvme0n1p3 context.
  # Releasing device-mapper backend.
  # Closing read only fd for /dev/nvme0n1p3.
  # Unlocking memory.
  Command failed with code -2 (no permission or bad passphrase).

  echo 123456 | cryptsetup open --type luks --debug /dev/nvme0n1p3 nvme0n1p3_crypt
  # cryptsetup 2.4.3 processing "cryptsetup open --type luks --debug /dev/nvme0n1p3 nvme0n1p3_crypt"
  # Running command open.
  # Locking memory.
  # Installing SIGINT/SIGTERM handler.
  # Unblocking interruption on signal.
  # Allocating context for crypt device /dev/nvme0n1p3.
  # Trying to open and read device /dev/nvme0n1p3 with direct-io.
  # Initialising device-mapper backend library.
  # Trying to load any crypt type from device /dev/nvme0n1p3.
  # Crypto backend (OpenSSL 3.0.2 15 Mar 2022 [default]) initialized in cryptsetup library version 2.4.3.
  # Detected kernel Linux 5.15.0-46-generic x86_64.
  # Loading LUKS2 header (repair disabled).
  # Acquiring read lock for device /dev/nvme0n1p3.
  # Opening lock resource file /run/cryptsetup/L_259:3
  # Verifying lock handle for /dev/nvme0n1p3.
  # Device /dev/nvme0n1p3 READ lock taken.
  # Trying to read primary LUKS2 header at offset 0x0.
  # Opening locked device /dev/nvme0n1p3
  # Verifying locked device handle (bdev)
  # LUKS2 header version 2 of size 16384 bytes, checksum sha256.
  # Checksum:99172356e66a2fec247b1e5c758af8bc1338a3fb8bd973aab5e1512a93b2dbdc (on-disk)
  # Checksum:99172356e66a2fec247b1e5c758af8bc1338a3fb8bd973aab5e1512a93b2dbdc (in-memory)
  # Trying to read secondary LUKS2 header at offset 0x4000.
  # Reusing open ro fd on device /dev/nvme0n1p3
  # LUKS2 header version 2 of size 16384 bytes, checksum sha256.
  # Checksum:655a50aef64b6fd4e10b8863df72d7fc62a7020f74684cb3419d4e22adb6fd9c (on-disk)
  # Checksum:655a50aef64b6fd4e10b8863df72d7fc62a7020f74684cb3419d4e22adb6fd9c (in-memory)
  # Device size 497776852992, offset 16777216.
  # Device /dev/nvme0n1p3 READ lock released.
  # PBKDF argon2id, time_ms 2000 (iterations 0), max_memory_kb 1048576, parallel_threads 4.
  # Activating volume nvme0n1p3_crypt using token (any type) -1.
  # dm version   [ opencount flush ]   [16384] (*1)
  # dm versions   [ opencount flush ]   [16384] (*1)
  # Detected dm-ioctl version 4.45.0.
  # Detected dm-crypt version 1.23.0.
  # Device-mapper backend running with UDEV support enabled.
  # dm status nvme0n1p3_crypt  [ opencount noflush ]   [16384] (*1)
  No usable token is available.
  # STDIN descriptor passphrase entry requested.
  # Activating volume nvme0n1p3_crypt [keyslot -1] using passphrase.
  # dm versions   [ opencount flush ]   [16384] (*1)
  # dm status nvme0n1p3_crypt  [ opencount noflush ]   [16384] (*1)
  # Keyslot 0 priority 1 != 2 (required), skipped.
  # Keyslot 1 priority 1 != 2 (required), skipped.
  # Trying to open LUKS2 keyslot 0.
  # Running keyslot key derivation.
  # Reading keyslot area [0x8000].
  # Acquiring read lock for device /dev/nvme0n1p3.
  # Opening lock resource file /run/cryptsetup/L_259:3
  # Verifying lock handle for /dev/nvme0n1p3.
  # Device /dev/nvme0n1p3 READ lock taken.
  # Reusing open ro fd on device /dev/nvme0n1p3
  # Device /dev/nvme0n1p3 READ lock released.
  # Verifying key from keyslot 0, digest 0.
  # Loading key (64 bytes, type logon) in thread keyring.
  # dm versions   [ opencount flush ]   [16384] (*1)
  # dm status nvme0n1p3_crypt  [ opencount noflush ]   [16384] (*1)
  # Calculated device size is 972187648 sectors (RW), offset 32768.
  # DM-UUID is CRYPT-LUKS2-a41dd71155cc4b6ca29d391c500c546d-nvme0n1p3_crypt
  # Udev cookie 0xd4d65d7 (semid 0) created
  # Udev cookie 0xd4d65d7 (semid 0) incremented to 1
  # Udev cookie 0xd4d65d7 (semid 0) incremented to 2
  # Udev cookie 0xd4d65d7 (semid 0) assigned to CREATE task(0) with flags DISABLE_LIBRARY_FALLBACK         (0x20)
  # dm create nvme0n1p3_crypt CRYPT-LUKS2-a41dd71155cc4b6ca29d391c500c546d-nvme0n1p3_crypt [ opencount flush ]   [16384] (*1)
  # dm reload   (253:0) [ opencount flush securedata ]   [16384] (*1)
  # dm resume nvme0n1p3_crypt  [ opencount flush securedata ]   [16384] (*1)
  # nvme0n1p3_crypt: Stacking NODE_ADD (253,0) 0:6 0660 [trust_udev]
  # nvme0n1p3_crypt: Stacking NODE_READ_AHEAD 256 (flags=1)
  # Udev cookie 0xd4d65d7 (semid 0) decremented to 1
  # Udev cookie 0xd4d65d7 (semid 0) waiting for zero
  # Udev cookie 0xd4d65d7 (semid 0) destroyed
  # nvme0n1p3_crypt: Skipping NODE_ADD (253,0) 0:6 0660 [trust_udev]
  # nvme0n1p3_crypt: Processing NODE_READ_AHEAD 256 (flags=1)
  # nvme0n1p3_crypt (253:0): read ahead is 256
  # nvme0n1p3_crypt: retaining kernel read ahead of 256 (requested 256)
  Key slot 0 unlocked.
  # Releasing crypt device /dev/nvme0n1p3 context.
  # Releasing device-mapper backend.
  # Closing read only fd for /dev/nvme0n1p3.
  # Unlocking memory.
  Command successful.

  Release and package version below:

  lsb_release -rd
  Description:	Ubuntu 22.04.1 LTS
  Release:	22.04

  apt-cache policy cryptsetup
  cryptsetup:
    Installed: 2:2.4.3-1ubuntu1.1
    Candidate: 2:2.4.3-1ubuntu1.1
    Version table:
   *** 2:2.4.3-1ubuntu1.1 500
          500 http://de.archive.ubuntu.com/ubuntu jammy-updates/main amd64 Packages
          100 /var/lib/dpkg/status
       2:2.4.3-1ubuntu1 500
          500 http://de.archive.ubuntu.com/ubuntu jammy/main amd64 Packages
  --- 
  ProblemType: Bug
  ApportVersion: 2.20.11-0ubuntu82.1
  Architecture: amd64
  CasperMD5CheckResult: pass
  DistroRelease: Ubuntu 22.04
  InstallationDate: Installed on 2022-08-15 (1 days ago)
  InstallationMedia: Ubuntu 22.04.1 LTS "Jammy Jellyfish" - Release amd64 (20220809.1)
  Package: linux
  PackageArchitecture: amd64
  ProcVersionSignature: Ubuntu 5.15.0-46.49-generic 5.15.39
  Tags:  jammy
  Uname: Linux 5.15.0-46-generic x86_64
  UpgradeStatus: No upgrade log present (probably fresh install)
  UserGroups: N/A
  _MarkForUpload: True
  cmdline: BOOT_IMAGE=/vmlinuz-5.15.0-46-generic root=/dev/mapper/vgubuntu-root ro quiet splash break=mountroot
  crypttab: nvme0n1p3_crypt UUID=a41dd711-55cc-4b6c-a29d-391c500c546d none luks,discard

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/cryptsetup/+bug/1986623/+subscriptions