[Bug 1820789] Re: Removing a hypervisor doesn't delete it entirely

[Launchpad Bug Tracker]

Status changed to 'Confirmed' because the bug affects multiple users.

** Changed in: nova (Ubuntu)
       Status: New => Confirmed

-- 
You received this bug notification because you are a member of Ubuntu
OpenStack, which is subscribed to nova in Ubuntu.
https://bugs.launchpad.net/bugs/1820789

Title:
  Removing a hypervisor doesn't delete it entirely

Status in nova package in Ubuntu:
  Confirmed

Bug description:
  When removing a host (because it got rebuilt, for example), we use:

  openstack compute service delete <someid>

  I expected that to remove the hostname cleanly from the database (or
  at least mark it as deleted) so that the hostname can be re-used.
  This isn't the case, the host remained in the nova_api database in the
  resource_providers table, and therefore could not be re-used.

  Starting nova-compute on the host in this state resulted in:

  2019-03-18 22:48:26.023 62597 ERROR nova.scheduler.client.report [req-445f587d-74e5-4a96-a5b5-4717f9095fb6 - - - - -] [req-1f1e781e-2ed8-4f6a-ac9e-93ecc450cec5] Failed to create resource provider record in placement API for UUID c6c4d923-1d0c-4f12-8505-d5af60c28ade. Got 409: {"errors": [{"status": 409, "request_id": "req-1f1e781e-2ed8-4f6a-ac9e-93ecc450cec5", "detail": "There was a conflict when trying to complete your request.\n\n Conflicting resource provider name: host22.maas already exists.  ", "title": "Conflict"}]}.
  2019-03-18 22:48:26.024 62597 ERROR nova.compute.manager [req-445f587d-74e5-4a96-a5b5-4717f9095fb6 - - - - -] Error updating resources for node host22.maas.: ResourceProviderCreationFailed: Failed to create resource provider host22.maas
  2019-03-18 22:48:26.024 62597 ERROR nova.compute.manager Traceback (most recent call last):
  2019-03-18 22:48:26.024 62597 ERROR nova.compute.manager   File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 7345, in update_available_resource_for_node
  2019-03-18 22:48:26.024 62597 ERROR nova.compute.manager     rt.update_available_resource(context, nodename)
  2019-03-18 22:48:26.024 62597 ERROR nova.compute.manager   File "/usr/lib/python2.7/dist-packages/nova/compute/resource_tracker.py", line 689, in update_available_resource
  2019-03-18 22:48:26.024 62597 ERROR nova.compute.manager     self._update_available_resource(context, resources)
  2019-03-18 22:48:26.024 62597 ERROR nova.compute.manager   File "/usr/lib/python2.7/dist-packages/oslo_concurrency/lockutils.py", line 277, in inner
  2019-03-18 22:48:26.024 62597 ERROR nova.compute.manager     return f(*args, **kwargs)
  2019-03-18 22:48:26.024 62597 ERROR nova.compute.manager   File "/usr/lib/python2.7/dist-packages/nova/compute/resource_tracker.py", line 713, in _update_available_resource
  2019-03-18 22:48:26.024 62597 ERROR nova.compute.manager     self._init_compute_node(context, resources)
  2019-03-18 22:48:26.024 62597 ERROR nova.compute.manager   File "/usr/lib/python2.7/dist-packages/nova/compute/resource_tracker.py", line 572, in _init_compute_node
  2019-03-18 22:48:26.024 62597 ERROR nova.compute.manager     self._update(context, cn)
  2019-03-18 22:48:26.024 62597 ERROR nova.compute.manager   File "/usr/lib/python2.7/dist-packages/nova/compute/resource_tracker.py", line 887, in _update
  2019-03-18 22:48:26.024 62597 ERROR nova.compute.manager     inv_data,
  2019-03-18 22:48:26.024 62597 ERROR nova.compute.manager   File "/usr/lib/python2.7/dist-packages/nova/scheduler/client/__init__.py", line 68, in set_inventory_for_provider
  2019-03-18 22:48:26.024 62597 ERROR nova.compute.manager     parent_provider_uuid=parent_provider_uuid,
  2019-03-18 22:48:26.024 62597 ERROR nova.compute.manager   File "/usr/lib/python2.7/dist-packages/nova/scheduler/client/__init__.py", line 37, in __run_method
  2019-03-18 22:48:26.024 62597 ERROR nova.compute.manager     return getattr(self.instance, __name)(*args, **kwargs)
  2019-03-18 22:48:26.024 62597 ERROR nova.compute.manager   File "/usr/lib/python2.7/dist-packages/nova/scheduler/client/report.py", line 1104, in set_inventory_for_provider
  2019-03-18 22:48:26.024 62597 ERROR nova.compute.manager     parent_provider_uuid=parent_provider_uuid)
  2019-03-18 22:48:26.024 62597 ERROR nova.compute.manager   File "/usr/lib/python2.7/dist-packages/nova/scheduler/client/report.py", line 665, in _ensure_resource_provider
  2019-03-18 22:48:26.024 62597 ERROR nova.compute.manager     parent_provider_uuid=parent_provider_uuid)
  2019-03-18 22:48:26.024 62597 ERROR nova.compute.manager   File "/usr/lib/python2.7/dist-packages/nova/scheduler/client/report.py", line 64, in wrapper
  2019-03-18 22:48:26.024 62597 ERROR nova.compute.manager     return f(self, *a, **k)
  2019-03-18 22:48:26.024 62597 ERROR nova.compute.manager   File "/usr/lib/python2.7/dist-packages/nova/scheduler/client/report.py", line 612, in _create_resource_provider
  2019-03-18 22:48:26.024 62597 ERROR nova.compute.manager     raise exception.ResourceProviderCreationFailed(name=name)
  2019-03-18 22:48:26.024 62597 ERROR nova.compute.manager ResourceProviderCreationFailed: Failed to create resource provider host22.maas
  2019-03-18 22:48:26.024 62597 ERROR nova.compute.manager 

  I was unable to clear the database entry:
  mysql> delete from resource_providers where name='host22.maas';
  ERROR 1451 (23000): Cannot delete or update a parent row: a foreign key constraint fails (`nova_api`.`resource_providers`, CONSTRAINT `resource_providers_ibfk_1` FOREIGN KEY (`root_provider_id`) REFERENCES `resource_providers` (`id`))

  I did, in the end, rename the host info in the table which allowed me to continue:
  update resource_providers set name='host22.maas.deleted' where id='94';

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/nova/+bug/1820789/+subscriptions