[Bug 1809454] Re: [SRU] nova rbd auth fallback uses cinder user with libvirt secret

[Brian Murray]

Hello Corey, or anyone else affected,

Accepted nova into cosmic-proposed. The package will build now and be
available at https://launchpad.net/ubuntu/+source/nova/2:18.0.3-0ubuntu2
in a few hours, and then in the -proposed repository.

Please help us by testing this new package.  See
https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how
to enable and use -proposed.  Your feedback will aid us getting this
update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug,
mentioning the version of the package you tested and change the tag from
verification-needed-cosmic to verification-done-cosmic. If it does not
fix the bug for you, please add a comment stating that, and change the
tag to verification-failed-cosmic. In either case, without details of
your testing we will not be able to proceed.

Further information regarding the verification process can be found at
https://wiki.ubuntu.com/QATeam/PerformingSRUVerification .  Thank you in
advance for helping!

N.B. The updated package will be released to -updates after the bug(s)
fixed by this package have been verified and the package has been in
-proposed for a minimum of 7 days.

** Changed in: nova (Ubuntu Cosmic)
       Status: Triaged => Fix Committed

** Tags added: verification-needed verification-needed-cosmic

-- 
You received this bug notification because you are a member of Ubuntu
OpenStack, which is subscribed to Ubuntu Cloud Archive.
https://bugs.launchpad.net/bugs/1809454

Title:
  [SRU] nova rbd auth fallback uses cinder user with libvirt secret

Status in Ubuntu Cloud Archive:
  Triaged
Status in Ubuntu Cloud Archive ocata series:
  Triaged
Status in Ubuntu Cloud Archive pike series:
  Triaged
Status in Ubuntu Cloud Archive queens series:
  Fix Committed
Status in Ubuntu Cloud Archive rocky series:
  Fix Committed
Status in Ubuntu Cloud Archive stein series:
  Triaged
Status in OpenStack Compute (nova):
  Fix Released
Status in OpenStack Compute (nova) ocata series:
  Triaged
Status in OpenStack Compute (nova) pike series:
  Triaged
Status in OpenStack Compute (nova) queens series:
  In Progress
Status in OpenStack Compute (nova) rocky series:
  Fix Released
Status in nova package in Ubuntu:
  Fix Released
Status in nova source package in Bionic:
  Triaged
Status in nova source package in Cosmic:
  Fix Committed
Status in nova source package in Disco:
  Fix Released

Bug description:
  [Impact]
  From David Ames (thedac), originally posted to https://bugs.launchpad.net/charm-nova-compute/+bug/1671422/comments/25:

  Updating this bug. We may decide to move this elsewhere it at some
  point.

  We have a deployment that was upgraded through to pike at which point
  it was noticed that nova instances with ceph backed volumes would not
  start.

  The cinder key was manually added to the nova-compute nodes in /etc/ceph and with:
  sudo virsh secret-define --file /tmp/cinder.secret

  However, this did not resolve the problem. It appeared libvirt was
  trying to use a mixed pair of usernames and keys. It was using the
  cinder username but the nova-compute key.

  Looking at nova's code it falls back to nova.conf when it does not have a secret_uuid from cinder but it was not setting the username correctly.
  https://github.com/openstack/nova/blob/stable/pike/nova/virt/libvirt/volume/net.py#L74

  The following seems to mitigate this as a temporary fix on nova-
  compute until we can come up with a complete plan:

  https://pastebin.ubuntu.com/p/tGm7C7fpXT/

  diff --git a/nova/virt/libvirt/volume/net.py b/nova/virt/libvirt/volume/net.py
  index cec43ce93b..8b0148df0b 100644
  --- a/nova/virt/libvirt/volume/net.py
  +++ b/nova/virt/libvirt/volume/net.py
  @@ -71,6 +71,7 @@ class LibvirtNetVolumeDriver(libvirt_volume.LibvirtBaseVolumeDriver):
               else:
                   LOG.debug('Falling back to Nova configuration for RBD auth '
                             'secret_uuid value.')
                 + conf.auth_username = CONF.libvirt.rbd_user
                   conf.auth_secret_uuid = CONF.libvirt.rbd_secret_uuid
               # secret_type is always hard-coded to 'ceph' in cinder
               conf.auth_secret_type = netdisk_properties['secret_type']

  Apply to /usr/lib/python2.7/dist-
  packages/nova/virt/libvirt/volume/net.py

  We still need a migration plan to get from the topology with nova-
  compute directly related to ceph to the topology with cinder-ceph
  related to nova-compute using ceph-access which would populate
  cinder's secret_uuid.

  It is possible we will need to carry the patch for existing instances.
  It may be worth getting that upstream as master has the same problem.

  [Test Case]
  Upgrade a juju-deployed cloud with ceph backend for nova and cinder from pre-ocata to ocata or above. Ensure that nova instances with ceph backed volumes successfully start.

  [Regression Potential]
  The fix is minimal and will not be fixed in Ubuntu until it has been approved upstream.

To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-archive/+bug/1809454/+subscriptions