[Bug 1339609] [NEW] md raid resync causing KVM guests overload

[Stanislav Bocinec]

Public bug reported:

Everytime  /usr/share/mdadm/checkarray RAID consistency check is being
executed (started on first sunday of a month by /etc/cron.d/mdadm) on
Ubuntu 12.04.4 KVM hypervisor, KVM guests running on the host
(Ubuntu/Debian/Scientific linux/Windows ) start to swap, freeze,
throwing 'BUG: soft lockup - CPU stuck for..!' errors on the consoles,
processes are coredumping on the guests etc. KVM host starts to swap
even it has enough memory available and every disk IO demanding
operation/process on any guest causes whole environment is affected and
unusable (similar situation to
https://bugs.launchpad.net/ubuntu/+source/linux/+bug/312163).

RAID check procedure shouldn't cause the KVM guests become unusable.

Some of the KVM guests use either qcow2 image storage format (virtio) or
RAW disks (LVM, virtio). All KVM guests are affected.

Here is the system information:
root at kvmserver:~# lsb_release -rd
Description:    Ubuntu 12.04.4 LTS
Release:        12.04
root at kvmserver:~# uname -a
Linux kvmserver 3.2.0-59-generic #90-Ubuntu SMP Tue Jan 7 22:43:51 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux

HW: Dell PowerEdge R410, 2x Intel Xeon X5650, 64GB RAM, 4x2TB SATA 6.0Gb drives in following RAID configuration:
# cat /proc/mdstat 
Personalities : [raid1] [linear] [multipath] [raid0] [raid6] [raid5] [raid4] [raid10] 
md2 : active raid1 sdd3[2] sdc3[0]
      1924018648 blocks super 1.2 [2/2] [UU]
      
md1 : active raid1 sdd2[0] sdc2[1]
      29296576 blocks [4/2] [UU__]
      
md127 : active raid1 sde1[2] sdf1[0]
      2147481446 blocks super 1.2 [2/2] [UU]

# free -m
             total       used       free     shared    buffers     cached
Mem:         64419      56859       7559          0      11671       1185
-/+ buffers/cache:      44002      20416
Swap:        18335       5554      12781

root at kvmserver:~# dpkg -l |egrep 'qemu-kvm|mdadm'
ii  kvm                                  1:84+dfsg-0ubuntu16+1.2.0+dfsg+0~12.04~ppa0 dummy transitional package from kvm to qemu-kvm
ii  mdadm                                3.2.5-1ubuntu0.3                            tool to administer Linux MD arrays (software RAID)
ii  qemu                                 1.2.0+dfsg-0~12.04~ppa0                     dummy transitional package from qemu to qemu-kvm
ii  qemu-kvm                             1.2.0+dfsg-0~12.04~ppa0                     Full virtualization on supported hardware

** Affects: qemu-kvm (Ubuntu)
     Importance: Undecided
         Status: New

-- 
You received this bug notification because you are a member of Ubuntu
Server Team, which is subscribed to qemu-kvm in Ubuntu.
https://bugs.launchpad.net/bugs/1339609

Title:
  md raid resync causing KVM guests overload

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/qemu-kvm/+bug/1339609/+subscriptions