[Bug 1640676] [NEW] [SRU] libvirt 1.2.12 live-migration corrupts some instances

[Launchpad Bug Tracker]

You have been subscribed to a public bug by Hua Zhang (zhhuabj):

[Impact]

While memory load is high, libvirt 1.2.12 (kilo) live-migration corrupts
some instances

[Test Case]

We can replicate the corruption pretty much at will. The sequence of
events to trigger it is:

Create an instance using a cloud image
Start a job running with the following command: "dd if=/dev/urandom of=/var/tmp/mjb.1 bs=4M count=1000"
Live migrate the instance using a command like: "nova live-migration --block-migrate <server-id> <target-hypervisor>"
Once the migration has finished, stop the dd job on the instance
do a "Hard reboot" of the instance (eg: for openstack, nova reboot --hard $INSTANCE)
When the instance boots, file system corruption will be observed and it won't boot correctly

[Regression Potential]

[Other Info]

Both libvirt 1.2.16 (liberty) and libvirt 1.2.13 have already fixed this
problem. So this problem only happens on kilo.

Backported from upstream patches, before the commit 80c5f10e libvirt
just polls the events we are interested which can lead to drive mirror
can not be cancelled, then the destination is not in a consistent state.
in this case it is not safe to continue with the migration. so the
commit 80c5f10e introduces listening queue events instead of polling to
fix the problem.

http://libvirt.org/git/?p=libvirt.git;a=commitdiff;h=80c5f10e865cda0302519492f197cb020bd14a07
http://libvirt.org/git/?p=libvirt.git;a=commitdiff;h=76c61cdca20c106960af033e5d0f5da70177af0f
http://libvirt.org/git/?p=libvirt.git;a=commitdiff;h=c37943a0687a8fdb08e6eda8ae4b9f4f43f4f2ed
http://libvirt.org/git/?p=libvirt.git;a=commitdiff;h=c88b323bf5d5a070c074fda7adc11085f14415ce

BTW, we have completed 20 to 30 live migrations with I/O running and
have had no problems, and also tested that other functions continue to
work as expected.

** Affects: cloud-archive
     Importance: Undecided
         Status: New

** Affects: cloud-archive/kilo
     Importance: High
     Assignee: Hua Zhang (zhhuabj)
         Status: Triaged

** Affects: libvirt (Ubuntu)
     Importance: Undecided
         Status: Fix Released

** Affects: libvirt (Ubuntu Trusty)
     Importance: High
         Status: Triaged


** Tags: sts-sru
-- 
[SRU] libvirt 1.2.12 live-migration corrupts some instances
https://bugs.launchpad.net/bugs/1640676
You received this bug notification because you are a member of Ubuntu Sponsors Team, which is subscribed to the bug report.