[Bug 1771700] Related fix merged to nova (master)
OpenStack Infra
1771700 at bugs.launchpad.net
Thu Jun 28 12:23:41 UTC 2018
Reviewed: https://review.openstack.org/569149
Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=aa67211acb701f02937f6dfe58899be873c388ed
Submitter: Zuul
Branch: master
commit aa67211acb701f02937f6dfe58899be873c388ed
Author: Matt Riedemann <mriedem.os at gmail.com>
Date: Thu May 17 11:23:13 2018 -0400
Make nova-lvm run in check on libvirt changes and compute API tests
This changes the nova-lvm job to run in the check queue on libvirt
driver changes only, and only runs the tempest compute API tests
to save time since we don't need to run things like the cinder,
glance, neutron etc API tests.
Once we're comfortable with the stability of this job we can
make it voting and gating.
This is in response to bug 1771700 which could have been prevented
if we were gating on the nova-lvm job on libvirt changes.
Change-Id: Ieaf00bcb6cb885e544d05b6f7276b6470b123258
Related-Bug: #1771700
--
You received this bug notification because you are a member of Ubuntu
OpenStack, which is subscribed to Ubuntu Cloud Archive.
https://bugs.launchpad.net/bugs/1771700
Title:
nova-lvm tempest job failing with InvalidDiskInfo
Status in Ubuntu Cloud Archive:
Fix Committed
Status in Ubuntu Cloud Archive ocata series:
Fix Committed
Status in Ubuntu Cloud Archive pike series:
Fix Committed
Status in Ubuntu Cloud Archive queens series:
Fix Committed
Status in Ubuntu Cloud Archive rocky series:
Fix Committed
Status in OpenStack Compute (nova):
Fix Released
Status in OpenStack Compute (nova) ocata series:
Fix Committed
Status in OpenStack Compute (nova) pike series:
Fix Committed
Status in OpenStack Compute (nova) queens series:
Fix Committed
Status in nova package in Ubuntu:
Fix Released
Status in nova source package in Bionic:
Fix Committed
Status in nova source package in Cosmic:
Fix Released
Bug description:
There has been a recent regression in the nova-lvm tempest job. The
most recent passing run was on 2018-05-11 [1][2], so something
regressed it between then and yesterday 2018-05-15.
The build fails and the following trace is seen in the n-cpu log:
May 15 23:01:40.174233 ubuntu-xenial-rax-dfw-0004040560 nova-compute[28718]: ERROR nova.compute.manager Traceback (most recent call last):
May 15 23:01:40.174457 ubuntu-xenial-rax-dfw-0004040560 nova-compute[28718]: ERROR nova.compute.manager File "/opt/stack/new/nova/nova/compute/manager.py", line 7343, in update_available_resource_for_node
May 15 23:01:40.174699 ubuntu-xenial-rax-dfw-0004040560 nova-compute[28718]: ERROR nova.compute.manager rt.update_available_resource(context, nodename)
May 15 23:01:40.174922 ubuntu-xenial-rax-dfw-0004040560 nova-compute[28718]: ERROR nova.compute.manager File "/opt/stack/new/nova/nova/compute/resource_tracker.py", line 664, in update_available_resource
May 15 23:01:40.175170 ubuntu-xenial-rax-dfw-0004040560 nova-compute[28718]: ERROR nova.compute.manager resources = self.driver.get_available_resource(nodename)
May 15 23:01:40.175414 ubuntu-xenial-rax-dfw-0004040560 nova-compute[28718]: ERROR nova.compute.manager File "/opt/stack/new/nova/nova/virt/libvirt/driver.py", line 6391, in get_available_resource
May 15 23:01:40.175641 ubuntu-xenial-rax-dfw-0004040560 nova-compute[28718]: ERROR nova.compute.manager disk_over_committed = self._get_disk_over_committed_size_total()
May 15 23:01:40.175868 ubuntu-xenial-rax-dfw-0004040560 nova-compute[28718]: ERROR nova.compute.manager File "/opt/stack/new/nova/nova/virt/libvirt/driver.py", line 7935, in _get_disk_over_committed_size_total
May 15 23:01:40.176091 ubuntu-xenial-rax-dfw-0004040560 nova-compute[28718]: ERROR nova.compute.manager config, block_device_info)
May 15 23:01:40.176333 ubuntu-xenial-rax-dfw-0004040560 nova-compute[28718]: ERROR nova.compute.manager File "/opt/stack/new/nova/nova/virt/libvirt/driver.py", line 7852, in _get_instance_disk_info_from_config
May 15 23:01:40.176555 ubuntu-xenial-rax-dfw-0004040560 nova-compute[28718]: ERROR nova.compute.manager virt_size = disk_api.get_disk_size(path)
May 15 23:01:40.176773 ubuntu-xenial-rax-dfw-0004040560 nova-compute[28718]: ERROR nova.compute.manager File "/opt/stack/new/nova/nova/virt/disk/api.py", line 99, in get_disk_size
May 15 23:01:40.176994 ubuntu-xenial-rax-dfw-0004040560 nova-compute[28718]: ERROR nova.compute.manager return images.qemu_img_info(path).virtual_size
May 15 23:01:40.177215 ubuntu-xenial-rax-dfw-0004040560 nova-compute[28718]: ERROR nova.compute.manager File "/opt/stack/new/nova/nova/virt/images.py", line 87, in qemu_img_info
May 15 23:01:40.177452 ubuntu-xenial-rax-dfw-0004040560 nova-compute[28718]: ERROR nova.compute.manager raise exception.InvalidDiskInfo(reason=msg)
May 15 23:01:40.177674 ubuntu-xenial-rax-dfw-0004040560 nova-compute[28718]: ERROR nova.compute.manager InvalidDiskInfo: Disk info file is invalid: qemu-img failed to execute on /dev/stack-volumes-default/8a1d5912-13e1-4583-876e-a04396b6b712_disk : Unexpected error while running command.
May 15 23:01:40.177902 ubuntu-xenial-rax-dfw-0004040560 nova-compute[28718]: ERROR nova.compute.manager Command: /usr/bin/python -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /dev/stack-volumes-default/8a1d5912-13e1-4583-876e-a04396b6b712_disk --force-share
May 15 23:01:40.178118 ubuntu-xenial-rax-dfw-0004040560 nova-compute[28718]: ERROR nova.compute.manager Exit code: 1
May 15 23:01:40.178344 ubuntu-xenial-rax-dfw-0004040560 nova-compute[28718]: ERROR nova.compute.manager Stdout: u''
May 15 23:01:40.178989 ubuntu-xenial-rax-dfw-0004040560 nova-compute[28718]: ERROR nova.compute.manager Stderr: u"qemu-img: Could not open '/dev/stack-volumes-default/8a1d5912-13e1-4583-876e-a04396b6b712_disk': Could not open '/dev/stack-volumes-default/8a1d5912-13e1-4583-876e-a04396b6b712_disk': Permission denied\n"
I think the failure is related to this change that merged on
2018-05-15:
https://review.openstack.org/567899
which moved the call of disk_api.get_disk_size(path) to be called for
all disk types instead of only for qcow2 and ploop. Based on the
surrounding code, only lvm.get_volume_size(path) should be called in
the case of lvm.
[1] http://zuul.openstack.org/builds.html?job_name=nova-lvm
[2] https://review.openstack.org/567916
To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-archive/+bug/1771700/+subscriptions
More information about the Ubuntu-openstack-bugs
mailing list