[Bug 2030476] Re: [UBUNTU 22.04] Kdump broken for s390 on Ubuntu 22.04.2

Frank Heimes 2030476 at bugs.launchpad.net
Wed Aug 9 17:35:49 UTC 2023 

** Description changed:

  SRU Justification:
  ==================
  
  [ Impact ]
  
   * The 'kdump-config load' step fails on s390x,
     while trying to create a dump with:
     $ kdump-config load
       * Creating symlink /var/lib/kdump/vmlinuz
       * Creating symlink /var/lib/kdump/initrd.img
       Unknown rela relocation: 0x14 0x6bc0901c
       * failed to load kdump kernel
  
   * Hence it's not possible to take a dump on jammy/s390x
     using kdump (which makes use of kexec).
  
   * The relocation entries of type R_390_PLT32DBL, that are generated
     by gcc 11.3 (and higher) for the kdump purgatory code are silently
     ignored, which leads to this kdump kernel load failures.
  
   * The solution is simply to handle all R_390_PLT32DBL relocation entries
     just like the R_390_PC32DBL ones.
  
  [ Test Plan ]
  
   * Have an affected s390x system (LPAR, z/VM or KVM) installed
     with jammy or newer.
  
   * sudo apt install kdump-tools kexec-tools
     (ensure the patched kexec-tools package is pulled-in)
  
   * Select 'yes' on "Should kdump-tools be enabled by default?"
  
-  * In case of a notification about a "Modified configuration file"
-    chose "install the package maintainer's version" to get
-    the new zipl.conf with the crashkernel argument.
+  * In case of a notification about a "Modified configuration file"
+    chose "install the package maintainer's version" to get
+    the new zipl.conf with the crashkernel argument.
  
   * sudo reboot
  
   * optional: systemctl status kdump-tools
  
   * sudo kdump-config load # see particularly here if it fails.
-    Before reboot a valid output is:
-    " * no crashkernel= parameter in the kernel cmdline"
-    after reboot its:
-    " * Cannot change symbolic links when kdump is loaded"
-    but no crash.
+    Before reboot a valid output is:
+    " * no crashkernel= parameter in the kernel cmdline"
+    after reboot its:
+    " * Cannot change symbolic links when kdump is loaded"
+    but no crash.
  
   * optional: kdump-config show
-    (look for line "current state:    ready to kdump"
+    (look for line "current state:    ready to kdump"
  
   * and to complete the overall case, trigger a dump:
  
   * cat /proc/sys/kernel/sysrq
     127
-    (anything else than "0" is good here and means enabled, but better do:)
+    (anything else than "0" is good here and means enabled, but better do:)
  
   * sudo sysctl -w kernel.sysrq=1
  
   * sudo -s
     echo c > /proc/sysrq-trigger
-    (you'll likely see "client_loop: send disconnect: Broken pipe")
+    (you'll likely see "client_loop: send disconnect: Broken pipe")
  
   * Now watch the console output and wait until system is back up again
-    and you'll find the dump in /var/crash:
-    202308091605
-    kdump_lock
-    kexec_cmd
-    linux-image-6.2.0-26-generic-202308091605.crash
+    and you'll find the dump in /var/crash:
+    202308091605
+    kdump_lock
+    kexec_cmd
+    linux-image-6.2.0-26-generic-202308091605.crash
  
  [ Where problems could occur ]
  
-  * First of all the changes are purely s390x related.
-    In kernel it's in /arch/s390/kernel/machine_kexec_file.c
-    and in kexec-tools it's in arch/s390/kexec-elf-rel-s390.c only.
+  * First of all the changes are purely s390x related,
+    only arch/s390/kexec-elf-rel-s390.c is touched in kexec-tools.
  
-  * In kernel all relocation entries (r_type) are simply changed from
+  * In kexec-tools all R_390_PLT32DBL relocation entries are
+    handled in the same way like the R_390_PC32DBL entries:
+    -		else if (r_type == R_390_PC32DBL)
+    +		else if (r_type == R_390_PC32DBL || r_type == R_390_PLT32DBL)
+ 
+  * This is btw. similar on how this is handled in the kernel.
+    All relocation entries (r_type) are simply changed from
     R_390_PLT32DBL to R_390_PC32DBL.
     +               if (r_type == R_390_PLT32DBL)
     +                       r_type = R_390_PC32DBL;
-    This could become problematic _if_ a special handling of
+  
+  * This could become problematic _if_ a special handling of
     R_390_PLT32DBL would be needed (which is not the case here).
  
-  * In kexec-tools it's similar. By this change:
-    -		else if (r_type == R_390_PC32DBL)
-    +		else if (r_type == R_390_PC32DBL || r_type == R_390_PLT32DBL)
-    both R_390_PC32DBL and R_390_PLT32DBL are now treated the same way.
- 
-  * This works as long as no further differentiation between both
+  * So this works as long as no further differentiation between both
     relocation entries are needed.
  
  [ Other Info ]
  
   * Jammy does not have the kernel nor the kexec-tools patch included.
  
   * lunar (with kernel 6.2) and mantic (with kernel 6.5, let's say > 6.2)
     have the kernel patch already in, since it got upstream accepted with 5.19.
     So these only need the kexec-tools patch, because both are still on
     kexec-tools v2.0.22, but the patch is only upstream with v2.0.24 and newer.
+ 
+  * And even for Jammy the commit
+    "s390/kexec: handle R_390_PLT32DBL rela in arch_kexec_apply_relocations_add()"
+    got already cherry-picked (here: 8ca23f0691841e4d83722ec5b2020259dd5d786e
+    and incl. in jammy since Ubuntu-5.15.0-19.19).
+    This was done as part of "Jammy update: v5.15.15 upstream stable release"
+    based on LP#1958418, because the upstream commit 
+    abf0e8e4ef25478a4390115e6a953d589d1f9ffd was tagged for stable updates.
+ 
+  * So we can ignore the kernels of the affected Ubuntu releases,
+    since they all have the needed commits included,
+    and can focus on the exec-tools SRU only.
  __________
  
  == Reported by <MIHAJLOV at de.ibm.com> - 2023-08-07 03:10:48 ==
  
  ---Problem Description---
  Kdump broken for s390
  
  ---uname output---
  Linux m836037 5.15.0-78-generic #85-Ubuntu SMP Fri Jul 7 15:25:49 UTC 2023 s390x s390x s390x GNU/Linux
  
  Machine Type = 3906-704
  
  ---Debugger---
  A debugger is not configured
  
  ---Steps to Reproduce---
  1. apt install kdump-tools
     enable kexec and kdump
  2. reboot
  3. kdump-config load
  ->
   # kdump-config load
    * Creating symlink /var/lib/kdump/vmlinuz
    * Creating symlink /var/lib/kdump/initrd.img
   Unknown rela relocation: 0x14 0x6bc0901c
    * failed to load kdump kernel

-- 
You received this bug notification because you are a member of Ubuntu
Sponsors, which is subscribed to the bug report.
https://bugs.launchpad.net/bugs/2030476

Title:
  [UBUNTU 22.04] Kdump broken for s390 on Ubuntu 22.04.2

Status in Ubuntu on IBM z Systems:
  In Progress
Status in kexec-tools package in Ubuntu:
  In Progress
Status in kexec-tools source package in Jammy:
  In Progress
Status in kexec-tools source package in Lunar:
  In Progress
Status in kexec-tools source package in Mantic:
  In Progress

Bug description:
  SRU Justification:
  ==================

  [ Impact ]

   * The 'kdump-config load' step fails on s390x,
     while trying to create a dump with:
     $ kdump-config load
       * Creating symlink /var/lib/kdump/vmlinuz
       * Creating symlink /var/lib/kdump/initrd.img
       Unknown rela relocation: 0x14 0x6bc0901c
       * failed to load kdump kernel

   * Hence it's not possible to take a dump on jammy/s390x
     using kdump (which makes use of kexec).

   * The relocation entries of type R_390_PLT32DBL, that are generated
     by gcc 11.3 (and higher) for the kdump purgatory code are silently
     ignored, which leads to this kdump kernel load failures.

   * The solution is simply to handle all R_390_PLT32DBL relocation entries
     just like the R_390_PC32DBL ones.

  [ Test Plan ]

   * Have an affected s390x system (LPAR, z/VM or KVM) installed
     with jammy or newer.

   * sudo apt install kdump-tools kexec-tools
     (ensure the patched kexec-tools package is pulled-in)

   * Select 'yes' on "Should kdump-tools be enabled by default?"

   * In case of a notification about a "Modified configuration file"
     chose "install the package maintainer's version" to get
     the new zipl.conf with the crashkernel argument.

   * sudo reboot

   * optional: systemctl status kdump-tools

   * sudo kdump-config load # see particularly here if it fails.
     Before reboot a valid output is:
     " * no crashkernel= parameter in the kernel cmdline"
     after reboot its:
     " * Cannot change symbolic links when kdump is loaded"
     but no crash.

   * optional: kdump-config show
     (look for line "current state:    ready to kdump"

   * and to complete the overall case, trigger a dump:

   * cat /proc/sys/kernel/sysrq
     127
     (anything else than "0" is good here and means enabled, but better do:)

   * sudo sysctl -w kernel.sysrq=1

   * sudo -s
     echo c > /proc/sysrq-trigger
     (you'll likely see "client_loop: send disconnect: Broken pipe")

   * Now watch the console output and wait until system is back up again
     and you'll find the dump in /var/crash:
     202308091605
     kdump_lock
     kexec_cmd
     linux-image-6.2.0-26-generic-202308091605.crash

  [ Where problems could occur ]

   * First of all the changes are purely s390x related,
     only arch/s390/kexec-elf-rel-s390.c is touched in kexec-tools.

   * In kexec-tools all R_390_PLT32DBL relocation entries are
     handled in the same way like the R_390_PC32DBL entries:
     -		else if (r_type == R_390_PC32DBL)
     +		else if (r_type == R_390_PC32DBL || r_type == R_390_PLT32DBL)

   * This is btw. similar on how this is handled in the kernel.
     All relocation entries (r_type) are simply changed from
     R_390_PLT32DBL to R_390_PC32DBL.
     +               if (r_type == R_390_PLT32DBL)
     +                       r_type = R_390_PC32DBL;
   
   * This could become problematic _if_ a special handling of
     R_390_PLT32DBL would be needed (which is not the case here).

   * So this works as long as no further differentiation between both
     relocation entries are needed.

  [ Other Info ]

   * Jammy does not have the kernel nor the kexec-tools patch included.

   * lunar (with kernel 6.2) and mantic (with kernel 6.5, let's say > 6.2)
     have the kernel patch already in, since it got upstream accepted with 5.19.
     So these only need the kexec-tools patch, because both are still on
     kexec-tools v2.0.22, but the patch is only upstream with v2.0.24 and newer.

   * And even for Jammy the commit
     "s390/kexec: handle R_390_PLT32DBL rela in arch_kexec_apply_relocations_add()"
     got already cherry-picked (here: 8ca23f0691841e4d83722ec5b2020259dd5d786e
     and incl. in jammy since Ubuntu-5.15.0-19.19).
     This was done as part of "Jammy update: v5.15.15 upstream stable release"
     based on LP#1958418, because the upstream commit 
     abf0e8e4ef25478a4390115e6a953d589d1f9ffd was tagged for stable updates.

   * So we can ignore the kernels of the affected Ubuntu releases,
     since they all have the needed commits included,
     and can focus on the exec-tools SRU only.
  __________

  == Reported by <MIHAJLOV at de.ibm.com> - 2023-08-07 03:10:48 ==

  ---Problem Description---
  Kdump broken for s390

  ---uname output---
  Linux m836037 5.15.0-78-generic #85-Ubuntu SMP Fri Jul 7 15:25:49 UTC 2023 s390x s390x s390x GNU/Linux

  Machine Type = 3906-704

  ---Debugger---
  A debugger is not configured

  ---Steps to Reproduce---
  1. apt install kdump-tools
     enable kexec and kdump
  2. reboot
  3. kdump-config load
  ->
   # kdump-config load
    * Creating symlink /var/lib/kdump/vmlinuz
    * Creating symlink /var/lib/kdump/initrd.img
   Unknown rela relocation: 0x14 0x6bc0901c
    * failed to load kdump kernel

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu-z-systems/+bug/2030476/+subscriptions

More information about the Ubuntu-sponsors mailing list