[Bug 1825882] Re: [SRU] Virsh disk attach errors silently ignored
Łukasz Zemczak
1825882 at bugs.launchpad.net
Thu May 16 17:17:48 UTC 2019
Hello Gustavo, or anyone else affected,
Accepted nova into cosmic-proposed. The package will build now and be
available at https://launchpad.net/ubuntu/+source/nova/2:18.1.0-0ubuntu3
in a few hours, and then in the -proposed repository.
Please help us by testing this new package. See
https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how
to enable and use -proposed. Your feedback will aid us getting this
update out to other Ubuntu users.
If this package fixes the bug for you, please add a comment to this bug,
mentioning the version of the package you tested and change the tag from
verification-needed-cosmic to verification-done-cosmic. If it does not
fix the bug for you, please add a comment stating that, and change the
tag to verification-failed-cosmic. In either case, without details of
your testing we will not be able to proceed.
Further information regarding the verification process can be found at
https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in
advance for helping!
N.B. The updated package will be released to -updates after the bug(s)
fixed by this package have been verified and the package has been in
-proposed for a minimum of 7 days.
** Changed in: nova (Ubuntu Cosmic)
Status: Triaged => Fix Committed
** Tags added: verification-needed-cosmic
--
You received this bug notification because you are a member of Ubuntu
OpenStack, which is subscribed to Ubuntu Cloud Archive.
https://bugs.launchpad.net/bugs/1825882
Title:
[SRU] Virsh disk attach errors silently ignored
Status in Ubuntu Cloud Archive:
Triaged
Status in Ubuntu Cloud Archive queens series:
Triaged
Status in Ubuntu Cloud Archive rocky series:
Triaged
Status in Ubuntu Cloud Archive stein series:
Triaged
Status in OpenStack Compute (nova):
Fix Released
Status in OpenStack Compute (nova) queens series:
Fix Committed
Status in OpenStack Compute (nova) rocky series:
Fix Committed
Status in OpenStack Compute (nova) stein series:
Fix Committed
Status in nova package in Ubuntu:
Triaged
Status in nova source package in Bionic:
Triaged
Status in nova source package in Cosmic:
Fix Committed
Status in nova source package in Disco:
Fix Committed
Bug description:
[Impact]
The following commit (1) is causing volume attachments which fail due
to libvirt device attach erros to be silently ignored and Nova report
the attachment as successful.
It seems that the original intention of the commit was to log a
condition and re-raise the exeption, but if the exception is of type
libvirt.libvirtError and does not contain the searched pattern, the
exception is ignored. If you unindent the raise statement, errors are
reported again.
In our case we had ceph/apparmor configuration problems in compute
nodes which prevented virsh attaching the device; volumes appeared as
successfully attached but the corresponding block device missing in
guests VMs. Other libvirt attach error conditions are ignored also, as
when you have already occuppied device names (i.e. 'Target vdb already
exists', device is busy, etc.)
(1)
https://github.com/openstack/nova/commit/78891c2305bff6e16706339a9c5eca99a84e409c
[Test Case]
* Deploy any OpenStack version up to Pike , which includes ceph backed cinder
* Create a guest VM (openstack server ...)
* Create a test cinder volume
$ openstack volume create test --size 10
* Force a drop on ceph traffic. Run the following command on the nova
hypervisor on which the server runs.
$ iptables -A OUTPUT -d ceph-mon-addr -p tcp --dport 6800 -j DROP
* Attach the volume to a running instance.
$ openstack server add volume 7151f507-a6b7-4f6d-a4cc-fd223d9feb5d
742ff117-21ae-4d1b-a52b-5b37955716ff
* This should cause the volume attachment to fail
$ virsh domblklist instance-xxxxx
Target Source
------------------------------------------------
vda nova/7151f507-a6b7-4f6d-a4cc-fd223d9feb5d_disk
No volume should attached after this step.
* If the behavior is fixed:
* Check that openstack server show , doesn't displays the displays the volume as attached.
* Check that proper log entries states the libvirt exception and error.
* If the behavior isn't fixed:
* openstack server show <ID> , will display the volume in the
volumes_attached property.
[Expected result]
* Volume attach fails and a proper exception is logged.
[Actual result]
* Volume attach fails but remains connected to the host and no further
exception gets logged.
[Regression Potential]
* The patches have been cherry-picked from upstream which helps to
reduce the regression potential of these fixes.
[Other Info]
* N/A
Description
To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-archive/+bug/1825882/+subscriptions
More information about the Ubuntu-openstack-bugs
mailing list