[Bug 1925211] Re: Hot-unplug of disks leaves broken block devices around in Hirsute

Tue Apr 20 20:01:37 UTC 2021

commit 8cc0dcfdc1c0e0be107d0288f9c0cf1f4201be62
Author: Vineeth Vijayan <vneethv at linux.ibm.com>
Date:   Fri Nov 20 09:36:38 2020 +0100

    s390/cio: remove pm support from ccw bus driver

    As part of removing broken pm-support from s390 arch, remove
    the pm callbacks from ccw-bus driver.The power-management functions
    are unused since the 'commit 394216275c7d ("s390: remove broken
    hibernate / power management support")'.

    Signed-off-by: Vineeth Vijayan <vneethv at linux.ibm.com>
    Reviewed-by: Peter Oberparleiter <oberpar at linux.ibm.com>
    Signed-off-by: Heiko Carstens <hca at linux.ibm.com>

 arch/s390/include/asm/ccwdev.h |  10 --
 drivers/s390/cio/cmf.c         |   5 -
 drivers/s390/cio/device.c      | 247 +----------------------------------------
 drivers/s390/cio/device.h      |   1 -
 drivers/s390/cio/device_fsm.c  |   6 -
 drivers/s390/cio/io_sch.h      |   1 -

It seems it wasn't as unused/broken as they thought :-/
BTW the referenced 394216275c7d ("s390: remove broken hibernate / power management support") was in v5.7

Found by:

$ git bisect log
git bisect start
# good: [2c85ebc57b3e1817b6ce1a6b703928e113a90442] Linux 5.10
git bisect good 2c85ebc57b3e1817b6ce1a6b703928e113a90442
# bad: [f40ddce88593482919761f74910f42f4b84c004b] Linux 5.11
git bisect bad f40ddce88593482919761f74910f42f4b84c004b
# bad: [538fcf57aaee6ad78a05f52b69a99baa22b33418] Merge branches 'acpi-scan', 'acpi-pnp' and 'acpi-sleep'
git bisect bad 538fcf57aaee6ad78a05f52b69a99baa22b33418
# bad: [15b447361794271f4d03c04d82276a841fe06328] mm/lru: revise the comments of lru_lock
git bisect bad 15b447361794271f4d03c04d82276a841fe06328
# good: [b10733527bfd864605c33ab2e9a886eec317ec39] Merge tag 'amd-drm-next-5.11-2020-12-09' of git://people.freedesktop.org/~agd5f/linux into drm-next
git bisect good b10733527bfd864605c33ab2e9a886eec317ec39
# good: [2c075f38a708c578a752b738a45e8c26923eac2e] Merge branch 'radeon-fixes' (Radeon and amdgpu fixes)
git bisect good 2c075f38a708c578a752b738a45e8c26923eac2e
# bad: [76d4acf22b4847f6c7b2f9042366fbdc3d20f578] Merge tag 'perf-kprobes-2020-12-14' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip
git bisect bad 76d4acf22b4847f6c7b2f9042366fbdc3d20f578
# bad: [f9b4240b074730f41c1ef8e0d695d10fb5bb1e27] Merge tag 'fixes-v5.11' of git://git.kernel.org/pub/scm/linux/kernel/git/brauner/linux
git bisect bad f9b4240b074730f41c1ef8e0d695d10fb5bb1e27
# good: [d889797530c66f699170233474eab3361471e808] Merge remote-tracking branch 'arm64/for-next/fixes' into for-next/core
git bisect good d889797530c66f699170233474eab3361471e808
# good: [2f6ea6fb88ab9d517644a098fc670b4d5dd1735e] s390/tape: remove unsupported PM functions
git bisect good 2f6ea6fb88ab9d517644a098fc670b4d5dd1735e
# bad: [586592478b1fa8bb8cd6875a9191468e9b1a8b13] Merge tag 's390-5.11-1' of git://git.kernel.org/pub/scm/linux/kernel/git/s390/linux
git bisect bad 586592478b1fa8bb8cd6875a9191468e9b1a8b13
# good: [0b03beface02d519693edb8020f9811c67d5c88f] Merge tag 'm68k-for-v5.11-tag1' of git://git.kernel.org/pub/scm/linux/kernel/git/geert/linux-m68k
git bisect good 0b03beface02d519693edb8020f9811c67d5c88f
# bad: [613775d62ec60202f98d2c5f520e6e9ba6dd4ac4] s390/kexec_file: fix diag308 subcode when loading crash kernel
git bisect bad 613775d62ec60202f98d2c5f520e6e9ba6dd4ac4
# good: [796cfabde5b23e1e506994fd11d2609ec2818e8d] s390/cio: remove pm support from IO subchannel drivers
git bisect good 796cfabde5b23e1e506994fd11d2609ec2818e8d
# bad: [8cc0dcfdc1c0e0be107d0288f9c0cf1f4201be62] s390/cio: remove pm support from ccw bus driver
git bisect bad 8cc0dcfdc1c0e0be107d0288f9c0cf1f4201be62
# good: [ef2eea78a6f363a58c909b7a0fd002df7cc52991] s390/cio: remove pm support from css-bus driver
git bisect good ef2eea78a6f363a58c909b7a0fd002df7cc52991
# first bad commit: [8cc0dcfdc1c0e0be107d0288f9c0cf1f4201be62] s390/cio: remove pm support from ccw bus driver

-- 
You received this bug notification because you are a member of Ubuntu
Foundations Bugs, which is subscribed to systemd in Ubuntu.
https://bugs.launchpad.net/bugs/1925211

Title:
  Hot-unplug of disks leaves broken block devices around in Hirsute

Status in linux package in Ubuntu:
  Confirmed
Status in systemd package in Ubuntu:
  New
Status in udev package in Ubuntu:
  New
Status in linux source package in Hirsute:
  Confirmed
Status in systemd source package in Hirsute:
  New
Status in udev source package in Hirsute:
  New

Bug description:
  Repro:
  #1 Get a guest
  $ uvt-kvm create --disk 5  --password=ubuntu h release=hirsute arch=s390x label=daily
  $ uvt-kvm wait h release=hirsute arch=s390x label=daily

  #2 Attach and Detach disk
  $ sudo qemu-img create -f qcow2 /var/lib/libvirt/images/test.qcow2 10M
  $ virsh attach-disk h /var/lib/libvirt/images/test.qcow2 vdc
  $ virsh detach-disk h vdc

  From libvirts POV it is gone at this point
  $ virsh domblklist h
   Target   Source
  ------------------------------------------------------------------
   vda      /var/lib/uvtool/libvirt/images/hirsute-2nd-zfs.qcow
   vdb      /var/lib/uvtool/libvirt/images/hirsute-2nd-zfs-ds.qcow

  But the guest thinks still it is present
  $ uvt-kvm ssh --insecure hirsute-2nd-zfs lsblk
    ...
    vdc    252:32   0   20M  0 disk

  This even remains a while after (not a race).

  Any access to it in the guest will hang (as you'd expect of a non-existing blockdev)
  4     0    1758    1739  20   0  12140  4800 -      S+   pts/0      0:00  |           \_ sudo mkfs.ext4 /dev/vdc
  4     0    1759    1758  20   0   6924  1044 -      D+   pts/0      0:00  |               \_ mkfs.ext4 /dev/vdc

  The result above was originally found with hirsute-guest at hirsute-host
  on s390x

  I do NOT see the same with  groovy-guest at hirsute-host on s390x
  I DO see the same with hirsute-guest at groovy-host on s390x
    => Guest version dependent not Host/Hipervisor dependent
  I DO see the same with ZFS disks AND LVM disks being added&removed
    => not type dependent
  I do NOT see the same on x86.
    => Arch dependent ??

  ... the evidence slowly points towards an issue in the guest, damn we are so
  close to release - but non-fully detaching disks are critical in my POV :-/

  Filing this as-is for awareness, but certainly this will need more debugging.
  Unsure where this is going to eventually I'll now file it for kernel/udev/systemd.
  If there are any known issues/components that are related let me know please!
  --- 
  ProblemType: Bug
  AlsaDevices: Error: command ['ls', '-l', '/dev/snd/'] failed with exit code 2: ls: cannot access '/dev/snd/': No such file or directory
  AplayDevices: Error: [Errno 2] No such file or directory: 'aplay'
  ApportVersion: 2.20.11-0ubuntu65
  Architecture: s390x
  ArecordDevices: Error: [Errno 2] No such file or directory: 'arecord'
  CRDA: N/A
  CasperMD5CheckResult: unknown
  DistroRelease: Ubuntu 21.04
  IwConfig: Error: [Errno 2] No such file or directory: 'iwconfig'
  Lspci:

  Lspci-vt: -[0000:00]-
  Lsusb: Error: command ['lsusb'] failed with exit code 1:
  Lsusb-t: Error: command ['lsusb', '-t'] failed with exit code 1: /sys/bus/usb/devices: No such file or directory
  Lsusb-v: Error: command ['lsusb', '-v'] failed with exit code 1:
  Package: udev
  PackageArchitecture: s390x
  PciMultimedia:

  ProcFB:

  ProcKernelCmdLine: root=LABEL=cloudimg-rootfs
  ProcVersionSignature: User Name 5.11.0-14.15-generic 5.11.12
  RelatedPackageVersions:
   linux-restricted-modules-5.11.0-14-generic N/A
   linux-backports-modules-5.11.0-14-generic  N/A
   linux-firmware                             N/A
  RfKill: Error: [Errno 2] No such file or directory: 'rfkill'
  Tags:  hirsute uec-images
  Uname: Linux 5.11.0-14-generic s390x
  UpgradeStatus: No upgrade log present (probably fresh install)
  UserGroups: adm audio cdrom dialout dip floppy lxd netdev plugdev sudo video
  _MarkForUpload: True
  acpidump:

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1925211/+subscriptions