[Bug 1948466] Re: [OVN] Mech driver fails to delete DHCP options during subnet deletion

[Rafael Lopez]

Hey Nikhil,

FWIW, I was able to get the test running using tox, and added your print
plus some code to the unit test to set empty subnet dict and verify the
fix (see attachment for test result).

ubuntu at sought-jackal:~/tmp/neutron$ git diff
diff --git a/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py b/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py
index 96c4457d19..ccf46481b1 100644
--- a/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py
+++ b/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py
@@ -1909,6 +1909,7 @@ class OVNClient(object):
         dhcp_options = self._nb_idl.get_subnet_dhcp_options(
             subnet_id, with_ports=True)

+        print(dhcp_options['subnet'])
         if dhcp_options['subnet']:
             txn.add(self._nb_idl.delete_dhcp_options(
                 dhcp_options['subnet']['uuid']))
diff --git a/neutron/tests/unit/plugins/ml2/drivers/ovn/mech_driver/test_mech_driver.py b/neutron/tests/unit/plugins/ml2/drivers/ovn/mech_driver/test_mech_driver.py
index bfd9100942..bd21a10488 100644
--- a/neutron/tests/unit/plugins/ml2/drivers/ovn/mech_driver/test_mech_driver.py
+++ b/neutron/tests/unit/plugins/ml2/drivers/ovn/mech_driver/test_mech_driver.py
@@ -1362,6 +1362,23 @@ class TestOVNMechanismDriver(test_plugin.Ml2PluginV2TestCase):
         self.mech_driver._nb_ovn.delete_dhcp_options.assert_has_calls(
             delete_dhcp_calls, any_order=True)

+        # reset call count
+        self.mech_driver._nb_ovn.delete_dhcp_options.call_count = 0
+        # test empty subnet
+        opts = {'subnet': {},
+                'ports': [{'uuid': 'port1-uuid'}]}
+        self.mech_driver._nb_ovn.get_subnet_dhcp_options.return_value = opts
+        self.mech_driver._ovn_client._remove_subnet_dhcp_options(
+            'subnet-id', mock.Mock())
+
+        # Check deleting DHCP_Options rows
+        # we expect call count of 1 since subnet is empty but ports isn't
+        delete_dhcp_calls = [mock.call('subnet-uuid'), mock.call('port1-uuid')]
+        self.assertEqual(1,
+            self.mech_driver._nb_ovn.delete_dhcp_options.call_count)
+        self.mech_driver._nb_ovn.delete_dhcp_options.assert_has_calls(
+            delete_dhcp_calls, any_order=True)
+
     def test_remove_subnet_dhcp_options_in_ovn_ipv4(self):
         self._test_remove_subnet_dhcp_options_in_ovn(4)

** Attachment added: "test_results_empty_dict.txt"
   https://bugs.launchpad.net/neutron/+bug/1948466/+attachment/5698112/+files/test_results_empty_dict.txt

-- 
You received this bug notification because you are a member of Ubuntu
OpenStack, which is subscribed to Ubuntu Cloud Archive.
https://bugs.launchpad.net/bugs/1948466

Title:
  [OVN] Mech driver fails to delete DHCP options during subnet deletion

Status in Ubuntu Cloud Archive:
  Fix Released
Status in Ubuntu Cloud Archive ussuri series:
  Fix Committed
Status in neutron:
  Fix Released
Status in neutron package in Ubuntu:
  Fix Released
Status in neutron source package in Focal:
  Fix Committed

Bug description:
  == Original Bug Description ==

  Snippet: https://paste.opendev.org/show/810168/

  I can't provide a link to a CI execution, I saw this error in an
  internal CI. I'm still investigating when this could happen.

  == Ubuntu SRU Details ==

  [Impact]

  During subnet deletion the check in _remove_subnet_dhcp_options()
  results in the following traceback (taken from pastebin above in case
  it disappears) if dhcp_options['subnet'] is an empty dictionary:

  ExternalNetworksRBACTestJSON-2078932943-project-admin] Mechanism driver 'ovn' failed in delete_subnet_postcommit: KeyError: 'uuid'
   Traceback (most recent call last):
     File "/opt/stack/neutron/neutron/plugins/ml2/managers.py", line 482, in _call_on_drivers
       getattr(driver.obj, method_name)(context)
     File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/mech_driver.py", line 637, in delete_subnet_postcommit
       self._ovn_client.delete_subnet(context._plugin_context,
     File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2103, in delete_subnet
       self._remove_subnet_dhcp_options(subnet_id, txn)
     File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 1971, in _remove_subnet_dhcp_options
       dhcp_options['subnet']['uuid']))
   KeyError: 'uuid'

  The fix ensures this check handles a dictionary correctly.

  [Test Case]

  In case we don't have a recreate for this:

  1) lxc launch ubuntu-daily:focal f1 && lxc exec f1 /bin/bash
  2) sudo add-apt-repository -p proposed
  3) sudo apt install python3-neutron
  4) cd /usr/lib/python3/dist-packages
  5) python3 -m unittest neutron.tests.unit.plugins.ml2.drivers.ovn.mech_driver.test_mech_driver.TestOVNMechanismDriver.test_remove_subnet_dhcp_options_in_ovn_ipv4
  6) re-run the test in step #5 after adding 'pdb.set_trace()' to the line before the check in neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py
     this way we can see what dhcp_options['subnet'] is set to, ensure the check behaves correctly, and try another run with dhcp_options['subnet'] = {}
  7) sudo add-apt-repository -r -p proposed

  [Regression Potential]

  This is a minimal change that is backward compatible with the previous
  check. The new check can still handle 'not None' in addition to
  handling an empty dictionary correctly. This has been fixed in Ubuntu
  Victoria packages (and above) since 2022-01-12, and has been fixed in
  the upstream stable/ussuri branch since 2021-10-25.

To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-archive/+bug/1948466/+subscriptions