[Bug 1771700] Related fix merged to nova (master)

Thu Jun 28 12:23:41 UTC 2018

Reviewed:  https://review.openstack.org/569149
Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=aa67211acb701f02937f6dfe58899be873c388ed
Submitter: Zuul
Branch:    master

commit aa67211acb701f02937f6dfe58899be873c388ed
Author: Matt Riedemann <mriedem.os at gmail.com>
Date:   Thu May 17 11:23:13 2018 -0400

    Make nova-lvm run in check on libvirt changes and compute API tests

    This changes the nova-lvm job to run in the check queue on libvirt
    driver changes only, and only runs the tempest compute API tests
    to save time since we don't need to run things like the cinder,
    glance, neutron etc API tests.

    Once we're comfortable with the stability of this job we can
    make it voting and gating.

    This is in response to bug 1771700 which could have been prevented
    if we were gating on the nova-lvm job on libvirt changes.

    Change-Id: Ieaf00bcb6cb885e544d05b6f7276b6470b123258
    Related-Bug: #1771700

-- 
You received this bug notification because you are a member of Ubuntu
OpenStack, which is subscribed to Ubuntu Cloud Archive.
https://bugs.launchpad.net/bugs/1771700

Title:
  nova-lvm tempest job failing with InvalidDiskInfo

Status in Ubuntu Cloud Archive:
  Fix Committed
Status in Ubuntu Cloud Archive ocata series:
  Fix Committed
Status in Ubuntu Cloud Archive pike series:
  Fix Committed
Status in Ubuntu Cloud Archive queens series:
  Fix Committed
Status in Ubuntu Cloud Archive rocky series:
  Fix Committed
Status in OpenStack Compute (nova):
  Fix Released
Status in OpenStack Compute (nova) ocata series:
  Fix Committed
Status in OpenStack Compute (nova) pike series:
  Fix Committed
Status in OpenStack Compute (nova) queens series:
  Fix Committed
Status in nova package in Ubuntu:
  Fix Released
Status in nova source package in Bionic:
  Fix Committed
Status in nova source package in Cosmic:
  Fix Released

Bug description:
  There has been a recent regression in the nova-lvm tempest job. The
  most recent passing run was on 2018-05-11 [1][2], so something
  regressed it between then and yesterday 2018-05-15.

  The build fails and the following trace is seen in the n-cpu log:

  May 15 23:01:40.174233 ubuntu-xenial-rax-dfw-0004040560 nova-compute[28718]: ERROR nova.compute.manager Traceback (most recent call last):
  May 15 23:01:40.174457 ubuntu-xenial-rax-dfw-0004040560 nova-compute[28718]: ERROR nova.compute.manager   File "/opt/stack/new/nova/nova/compute/manager.py", line 7343, in update_available_resource_for_node
  May 15 23:01:40.174699 ubuntu-xenial-rax-dfw-0004040560 nova-compute[28718]: ERROR nova.compute.manager     rt.update_available_resource(context, nodename)
  May 15 23:01:40.174922 ubuntu-xenial-rax-dfw-0004040560 nova-compute[28718]: ERROR nova.compute.manager   File "/opt/stack/new/nova/nova/compute/resource_tracker.py", line 664, in update_available_resource
  May 15 23:01:40.175170 ubuntu-xenial-rax-dfw-0004040560 nova-compute[28718]: ERROR nova.compute.manager     resources = self.driver.get_available_resource(nodename)
  May 15 23:01:40.175414 ubuntu-xenial-rax-dfw-0004040560 nova-compute[28718]: ERROR nova.compute.manager   File "/opt/stack/new/nova/nova/virt/libvirt/driver.py", line 6391, in get_available_resource
  May 15 23:01:40.175641 ubuntu-xenial-rax-dfw-0004040560 nova-compute[28718]: ERROR nova.compute.manager     disk_over_committed = self._get_disk_over_committed_size_total()
  May 15 23:01:40.175868 ubuntu-xenial-rax-dfw-0004040560 nova-compute[28718]: ERROR nova.compute.manager   File "/opt/stack/new/nova/nova/virt/libvirt/driver.py", line 7935, in _get_disk_over_committed_size_total
  May 15 23:01:40.176091 ubuntu-xenial-rax-dfw-0004040560 nova-compute[28718]: ERROR nova.compute.manager     config, block_device_info)
  May 15 23:01:40.176333 ubuntu-xenial-rax-dfw-0004040560 nova-compute[28718]: ERROR nova.compute.manager   File "/opt/stack/new/nova/nova/virt/libvirt/driver.py", line 7852, in _get_instance_disk_info_from_config
  May 15 23:01:40.176555 ubuntu-xenial-rax-dfw-0004040560 nova-compute[28718]: ERROR nova.compute.manager     virt_size = disk_api.get_disk_size(path)
  May 15 23:01:40.176773 ubuntu-xenial-rax-dfw-0004040560 nova-compute[28718]: ERROR nova.compute.manager   File "/opt/stack/new/nova/nova/virt/disk/api.py", line 99, in get_disk_size
  May 15 23:01:40.176994 ubuntu-xenial-rax-dfw-0004040560 nova-compute[28718]: ERROR nova.compute.manager     return images.qemu_img_info(path).virtual_size
  May 15 23:01:40.177215 ubuntu-xenial-rax-dfw-0004040560 nova-compute[28718]: ERROR nova.compute.manager   File "/opt/stack/new/nova/nova/virt/images.py", line 87, in qemu_img_info
  May 15 23:01:40.177452 ubuntu-xenial-rax-dfw-0004040560 nova-compute[28718]: ERROR nova.compute.manager     raise exception.InvalidDiskInfo(reason=msg)
  May 15 23:01:40.177674 ubuntu-xenial-rax-dfw-0004040560 nova-compute[28718]: ERROR nova.compute.manager InvalidDiskInfo: Disk info file is invalid: qemu-img failed to execute on /dev/stack-volumes-default/8a1d5912-13e1-4583-876e-a04396b6b712_disk : Unexpected error while running command.
  May 15 23:01:40.177902 ubuntu-xenial-rax-dfw-0004040560 nova-compute[28718]: ERROR nova.compute.manager Command: /usr/bin/python -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /dev/stack-volumes-default/8a1d5912-13e1-4583-876e-a04396b6b712_disk --force-share
  May 15 23:01:40.178118 ubuntu-xenial-rax-dfw-0004040560 nova-compute[28718]: ERROR nova.compute.manager Exit code: 1
  May 15 23:01:40.178344 ubuntu-xenial-rax-dfw-0004040560 nova-compute[28718]: ERROR nova.compute.manager Stdout: u''
  May 15 23:01:40.178989 ubuntu-xenial-rax-dfw-0004040560 nova-compute[28718]: ERROR nova.compute.manager Stderr: u"qemu-img: Could not open '/dev/stack-volumes-default/8a1d5912-13e1-4583-876e-a04396b6b712_disk': Could not open '/dev/stack-volumes-default/8a1d5912-13e1-4583-876e-a04396b6b712_disk': Permission denied\n"

  I think the failure is related to this change that merged on
  2018-05-15:

  https://review.openstack.org/567899

  which moved the call of disk_api.get_disk_size(path) to be called for
  all disk types instead of only for qcow2 and ploop. Based on the
  surrounding code, only lvm.get_volume_size(path) should be called in
  the case of lvm.

  [1] http://zuul.openstack.org/builds.html?job_name=nova-lvm
  [2] https://review.openstack.org/567916

To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-archive/+bug/1771700/+subscriptions