[Bug 1928031] Re: neutron-ovn-metadata-agent AttributeError: 'MetadataProxyHandler' object has no attribute 'sb_idl'
Frode Nordahl
1928031 at bugs.launchpad.net
Thu May 20 06:31:00 UTC 2021
Thanks for digging that up, does look relevant for sure. The linked
review appears to be backported to Ussuri already, and it is in the
neutron 16.3.2 point release (released 10 days ago). So we'll get that
in Ubuntu on the next point release update to the neutron package.
** Also affects: neutron (Ubuntu)
Importance: Undecided
Status: New
** Changed in: neutron (Ubuntu)
Status: New => Triaged
** Changed in: neutron (Ubuntu)
Importance: Undecided => High
** Changed in: charm-ovn-chassis
Status: New => Invalid
--
You received this bug notification because you are a member of Ubuntu
OpenStack, which is subscribed to neutron in Ubuntu.
https://bugs.launchpad.net/bugs/1928031
Title:
neutron-ovn-metadata-agent AttributeError: 'MetadataProxyHandler'
object has no attribute 'sb_idl'
Status in charm-ovn-chassis:
Invalid
Status in neutron package in Ubuntu:
Triaged
Bug description:
neutron-ovn-metadata-agent not able to handle any metadata requests
from the instances.
Scenario:
* Initially there is some intermittent connectivity issues that are descirbed in LP #1907686
https://bugs.launchpad.net/charm-ovn-chassis/+bug/1907686/comments/9
* The fix for the above is available in python3-openvswitch package in ussuri-proposed pocket
Installed the fix on all neutron-server and compute and restarted neutron-ovn-metadata-agent one by one
* neutron-ovn-metadata-agent on one of the compute nodes not able to
handle any metadata requests after restart. ( Please note the problem
happened with only one ovn-metadata agent and rest of the agents are
good on other compute nodes, so this is some race condition in IDL)
Stacktrace shows both the workers 69188/69189 timed out on OVNSB IDL
connection and hence sb_idl is never initialized.
Stacktrace of Attribute error:
------------------------------
2021-04-27 08:51:01.340 69188 ERROR neutron.agent.ovn.metadata.server Traceback (most recent call last):
2021-04-27 08:51:01.340 69188 ERROR neutron.agent.ovn.metadata.server File "/usr/lib/python3/dist-packages/neutron/agent/ovn/metadata/server.py", line 67, in __call__
2021-04-27 08:51:01.340 69188 ERROR neutron.agent.ovn.metadata.server instance_id, project_id = self._get_instance_and_project_id(req)
2021-04-27 08:51:01.340 69188 ERROR neutron.agent.ovn.metadata.server File "/usr/lib/python3/dist-packages/neutron/agent/ovn/metadata/server.py", line 84, in _get_instance_and_project_id
2021-04-27 08:51:01.340 69188 ERROR neutron.agent.ovn.metadata.server ports = self.sb_idl.get_network_port_bindings_by_ip(network_id,
2021-04-27 08:51:01.340 69188 ERROR neutron.agent.ovn.metadata.server AttributeError: 'MetadataProxyHandler' object has no attribute 'sb_idl'
2021-04-27 08:51:01.340 69188 ERROR neutron.agent.ovn.metadata.server
Stacktrace at the restart of neutron-ovn-metadata-agent process:
----------------------------------------------------------------
2021-04-15 22:27:03.803 69124 INFO neutron.common.config [-] /usr/bin/neutron-ovn-metadata-agent version 16.2.0
2021-04-15 22:27:03.832 69124 INFO ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: connecting...
2021-04-15 22:27:03.833 69124 INFO ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: connected
2021-04-15 22:27:03.949 69124 WARNING neutron.agent.ovn.metadata.agent [-] Can't read ovn-bridge external-id from OVSDB. Using br-int instead.
2021-04-15 22:27:03.950 69124 INFO oslo_service.service [-] Starting 2 workers
2021-04-15 22:27:03.985 69188 INFO ovsdbapp.backend.ovs_idl.vlog [-] ssl:10.216.241.118:6642: connecting...
2021-04-15 22:27:03.986 69189 INFO ovsdbapp.backend.ovs_idl.vlog [-] ssl:10.216.241.118:6642: connecting...
2021-04-15 22:27:04.005 69124 INFO ovsdbapp.backend.ovs_idl.vlog [-] ssl:10.216.241.118:6642: connecting...
2021-04-15 22:27:04.006 69188 INFO ovsdbapp.backend.ovs_idl.vlog [-] ssl:10.216.241.118:6642: connected
2021-04-15 22:27:04.033 69189 INFO ovsdbapp.backend.ovs_idl.vlog [-] ssl:10.216.241.118:6642: connected
2021-04-15 22:27:04.061 69124 INFO ovsdbapp.backend.ovs_idl.vlog [-] ssl:10.216.241.118:6642: connected
2021-04-15 22:27:06.129 69124 INFO oslo.privsep.daemon [-] Running privsep helper: ['sudo', '/usr/bin/neutron-rootwrap', '/etc/neutron/rootwrap.conf', 'privsep-helper', '--config-file', '/etc/neutron/neutron.conf', '--config-file', '/etc/neutron/neutron_ovn_metadata_agent.ini', '--privsep_context', 'neutron.privileged.default', '--privsep_sock_path', '/tmp/tmpgncr2rq7/privsep.sock']
2021-04-15 22:27:06.757 69124 INFO oslo.privsep.daemon [-] Spawned new privsep daemon via rootwrap
2021-04-15 22:27:06.676 69211 INFO oslo.privsep.daemon [-] privsep daemon starting
2021-04-15 22:27:06.678 69211 INFO oslo.privsep.daemon [-] privsep process running with uid/gid: 0/0
2021-04-15 22:27:06.680 69211 INFO oslo.privsep.daemon [-] privsep process running with capabilities (eff/prm/inh): CAP_DAC_OVERRIDE|CAP_DAC_READ_SEARCH|CAP_NET_ADMIN|CAP_SYS_ADMIN|CAP_SYS_PTRACE/CAP_DAC_OVERRIDE|CAP_DAC_READ_SEARCH|CAP_NET_ADMIN|CAP_SYS_ADMIN|CAP_SYS_PTRACE/none
2021-04-15 22:27:06.680 69211 INFO oslo.privsep.daemon [-] privsep daemon running as pid 69211
2021-04-15 22:30:08.542 69188 ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] OVS database connection to OVN_Southbound failed with error: 'Timeout'. Verify that the OVS and OVN services are available and that the 'ovn_nb_connection' and 'ovn_sb_connection' configuration options are correct.: Exception: Timeout
2021-04-15 22:30:08.542 69188 ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn Traceback (most recent call last):
2021-04-15 22:30:08.542 69188 ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn File "/usr/lib/python3/dist-packages/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 67, in start_connection
2021-04-15 22:30:08.542 69188 ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn self.ovsdb_connection.start()
2021-04-15 22:30:08.542 69188 ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn File "/usr/lib/python3/dist-packages/ovsdbapp/backend/ovs_idl/connection.py", line 79, in start
2021-04-15 22:30:08.542 69188 ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn idlutils.wait_for_change(self.idl, self.timeout)
2021-04-15 22:30:08.542 69188 ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn File "/usr/lib/python3/dist-packages/ovsdbapp/backend/ovs_idl/idlutils.py", line 173, in wait_for_change
2021-04-15 22:30:08.542 69188 ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn raise Exception("Timeout") # TODO(twilson) use TimeoutException?
2021-04-15 22:30:08.542 69188 ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn Exception: Timeout
2021-04-15 22:30:08.542 69188 ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn
2021-04-15 22:30:08.544 69188 ERROR neutron_lib.callbacks.manager [-] Error during notification for neutron.agent.ovn.metadata.server.MetadataProxyHandler.post_fork_initialize-476074 process, after_init: neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn.OvsdbConnectionUnavailable: OVS database connection to OVN_Southbound failed with error: 'Timeout'. Verify that the OVS and OVN services are available and that the 'ovn_nb_connection' and 'ovn_sb_connection' configuration options are correct.
2021-04-15 22:30:08.544 69188 ERROR neutron_lib.callbacks.manager Traceback (most recent call last):
2021-04-15 22:30:08.544 69188 ERROR neutron_lib.callbacks.manager File "/usr/lib/python3/dist-packages/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 67, in start_connection
2021-04-15 22:30:08.544 69188 ERROR neutron_lib.callbacks.manager self.ovsdb_connection.start()
2021-04-15 22:30:08.544 69188 ERROR neutron_lib.callbacks.manager File "/usr/lib/python3/dist-packages/ovsdbapp/backend/ovs_idl/connection.py", line 79, in start
2021-04-15 22:30:08.544 69188 ERROR neutron_lib.callbacks.manager idlutils.wait_for_change(self.idl, self.timeout)
2021-04-15 22:30:08.544 69188 ERROR neutron_lib.callbacks.manager File "/usr/lib/python3/dist-packages/ovsdbapp/backend/ovs_idl/idlutils.py", line 173, in wait_for_change
2021-04-15 22:30:08.544 69188 ERROR neutron_lib.callbacks.manager raise Exception("Timeout") # TODO(twilson) use TimeoutException?
2021-04-15 22:30:08.544 69188 ERROR neutron_lib.callbacks.manager Exception: Timeout
2021-04-15 22:30:08.544 69188 ERROR neutron_lib.callbacks.manager
2021-04-15 22:30:08.544 69188 ERROR neutron_lib.callbacks.manager During handling of the above exception, another exception occurred:
2021-04-15 22:30:08.544 69188 ERROR neutron_lib.callbacks.manager
2021-04-15 22:30:08.544 69188 ERROR neutron_lib.callbacks.manager Traceback (most recent call last):
2021-04-15 22:30:08.544 69188 ERROR neutron_lib.callbacks.manager
2021-04-15 22:30:08.544 69188 ERROR neutron_lib.callbacks.manager During handling of the above exception, another exception occurred:
2021-04-15 22:30:08.544 69188 ERROR neutron_lib.callbacks.manager
2021-04-15 22:30:08.544 69188 ERROR neutron_lib.callbacks.manager Traceback (most recent call last):
2021-04-15 22:30:08.544 69188 ERROR neutron_lib.callbacks.manager File "/usr/lib/python3/dist-packages/neutron_lib/callbacks/manager.py", line 197, in _notify_loop
2021-04-15 22:30:08.544 69188 ERROR neutron_lib.callbacks.manager callback(resource, event, trigger, **kwargs)
2021-04-15 22:30:08.544 69188 ERROR neutron_lib.callbacks.manager File "/usr/lib/python3/dist-packages/neutron/agent/ovn/metadata/server.py", line 60, in post_fork_initialize
2021-04-15 22:30:08.544 69188 ERROR neutron_lib.callbacks.manager tables=('Port_Binding', 'Datapath_Binding')).start()
2021-04-15 22:30:08.544 69188 ERROR neutron_lib.callbacks.manager File "/usr/lib/python3/dist-packages/neutron/agent/ovn/metadata/ovsdb.py", line 57, in start
2021-04-15 22:30:08.544 69188 ERROR neutron_lib.callbacks.manager return impl_idl_ovn.OvsdbSbOvnIdl(conn)
2021-04-15 22:30:08.544 69188 ERROR neutron_lib.callbacks.manager File "/usr/lib/python3/dist-packages/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 724, in __init__
2021-04-15 22:30:08.544 69188 ERROR neutron_lib.callbacks.manager super(OvsdbSbOvnIdl, self).__init__(connection)
2021-04-15 22:30:08.544 69188 ERROR neutron_lib.callbacks.manager File "/usr/lib/python3/dist-packages/ovsdbapp/schema/ovn_southbound/impl_idl.py", line 26, in __init__
2021-04-15 22:30:08.544 69188 ERROR neutron_lib.callbacks.manager super(OvnSbApiIdlImpl, self).__init__(connection)
2021-04-15 22:30:08.544 69188 ERROR neutron_lib.callbacks.manager File "/usr/lib/python3/dist-packages/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 63, in __init__
2021-04-15 22:30:08.544 69188 ERROR neutron_lib.callbacks.manager super(Backend, self).__init__(connection)
2021-04-15 22:30:08.544 69188 ERROR neutron_lib.callbacks.manager File "/usr/lib/python3/dist-packages/ovsdbapp/backend/ovs_idl/__init__.py", line 32, in __init__
2021-04-15 22:30:08.544 69188 ERROR neutron_lib.callbacks.manager self.start_connection(connection)
2021-04-15 22:30:08.544 69188 ERROR neutron_lib.callbacks.manager File "/usr/lib/python3/dist-packages/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 72, in start_connection
2021-04-15 22:30:08.544 69188 ERROR neutron_lib.callbacks.manager raise connection_exception
2021-04-15 22:30:08.544 69188 ERROR neutron_lib.callbacks.manager neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn.OvsdbConnectionUnavailable: OVS database connection to OVN_Southbound failed with error: 'Timeout'. Verify that the OVS and OVN services are available and that the 'ovn_nb_connection' and 'ovn_sb_connection' configuration options are correct.
2021-04-15 22:30:08.544 69188 ERROR neutron_lib.callbacks.manager
2021-04-15 22:30:08.544 69188 ERROR neutron_lib.callbacks.manager raise Exception("Timeout") # TODO(twilson) use TimeoutException?
2021-04-15 22:30:08.544 69188 ERROR neutron_lib.callbacks.manager Exception: Timeout
2021-04-15 22:30:08.544 69188 ERROR neutron_lib.callbacks.manager
2021-04-15 22:30:08.544 69188 ERROR neutron_lib.callbacks.manager During handling of the above exception, another exception occurred:
2021-04-15 22:30:08.544 69188 ERROR neutron_lib.callbacks.manager
2021-04-15 22:30:08.544 69188 ERROR neutron_lib.callbacks.manager Traceback (most recent call last):
2021-04-15 22:30:08.544 69188 ERROR neutron_lib.callbacks.manager File "/usr/lib/python3/dist-packages/neutron_lib/callbacks/manager.py", line 197, in _notify_loop
2021-04-15 22:30:08.544 69188 ERROR neutron_lib.callbacks.manager callback(resource, event, trigger, **kwargs)
2021-04-15 22:30:08.544 69188 ERROR neutron_lib.callbacks.manager File "/usr/lib/python3/dist-packages/neutron/agent/ovn/metadata/server.py", line 60, in post_fork_initialize
2021-04-15 22:30:08.544 69188 ERROR neutron_lib.callbacks.manager tables=('Port_Binding', 'Datapath_Binding')).start()
2021-04-15 22:30:08.544 69188 ERROR neutron_lib.callbacks.manager File "/usr/lib/python3/dist-packages/neutron/agent/ovn/metadata/ovsdb.py", line 57, in start
2021-04-15 22:30:08.544 69188 ERROR neutron_lib.callbacks.manager return impl_idl_ovn.OvsdbSbOvnIdl(conn)
2021-04-15 22:30:08.544 69188 ERROR neutron_lib.callbacks.manager File "/usr/lib/python3/dist-packages/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 724, in __init__
2021-04-15 22:30:08.544 69188 ERROR neutron_lib.callbacks.manager super(OvsdbSbOvnIdl, self).__init__(connection)
2021-04-15 22:30:08.544 69188 ERROR neutron_lib.callbacks.manager File "/usr/lib/python3/dist-packages/ovsdbapp/schema/ovn_southbound/impl_idl.py", line 26, in __init__
2021-04-15 22:30:08.544 69188 ERROR neutron_lib.callbacks.manager super(OvnSbApiIdlImpl, self).__init__(connection)
2021-04-15 22:30:08.544 69188 ERROR neutron_lib.callbacks.manager File "/usr/lib/python3/dist-packages/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 63, in __init__
2021-04-15 22:30:08.544 69188 ERROR neutron_lib.callbacks.manager super(Backend, self).__init__(connection)
2021-04-15 22:30:08.544 69188 ERROR neutron_lib.callbacks.manager File "/usr/lib/python3/dist-packages/ovsdbapp/backend/ovs_idl/__init__.py", line 32, in __init__
2021-04-15 22:30:08.544 69188 ERROR neutron_lib.callbacks.manager self.start_connection(connection)
2021-04-15 22:30:08.544 69188 ERROR neutron_lib.callbacks.manager File "/usr/lib/python3/dist-packages/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 72, in start_connection
2021-04-15 22:30:08.544 69188 ERROR neutron_lib.callbacks.manager raise connection_exception
2021-04-15 22:30:08.544 69188 ERROR neutron_lib.callbacks.manager neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn.OvsdbConnectionUnavailable: OVS database connection to OVN_Southbound failed with error: 'Timeout'. Verify that the OVS and OVN services are available and that the 'ovn_nb_connection' and 'ovn_sb_connection' configuration options are correct.
2021-04-15 22:30:08.544 69188 ERROR neutron_lib.callbacks.manager
2021-04-15 22:30:08.546 69188 INFO eventlet.wsgi.server [-] (69188) wsgi starting up on http:/var/lib/neutron/metadata_proxy
2021-04-15 22:30:08.550 69189 ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] OVS database connection to OVN_Southbound failed with error: 'Timeout'. Verify that the OVS and OVN services are available and that the 'ovn_nb_connection' and 'ovn_sb_connection' configuration options are correct.: Exception: Timeout
2021-04-15 22:30:08.550 69189 ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn Traceback (most recent call last):
2021-04-15 22:30:08.550 69189 ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn File "/usr/lib/python3/dist-packages/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 67, in start_connection
2021-04-15 22:30:08.550 69189 ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn self.ovsdb_connection.start()
2021-04-15 22:30:08.550 69189 ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn File "/usr/lib/python3/dist-packages/ovsdbapp/backend/ovs_idl/connection.py", line 79, in start
2021-04-15 22:30:08.550 69189 ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn idlutils.wait_for_change(self.idl, self.timeout)
2021-04-15 22:30:08.550 69189 ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn File "/usr/lib/python3/dist-packages/ovsdbapp/backend/ovs_idl/idlutils.py", line 173, in wait_for_change
2021-04-15 22:30:08.550 69189 ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn raise Exception("Timeout") # TODO(twilson) use TimeoutException?
2021-04-15 22:30:08.550 69189 ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn Exception: Timeout
2021-04-15 22:30:08.550 69189 ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn
2021-04-15 22:30:08.552 69189 ERROR neutron_lib.callbacks.manager [-] Error during notification for neutron.agent.ovn.metadata.server.MetadataProxyHandler.post_fork_initialize-476074 process, after_init: neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn.OvsdbConnectionUnavailable: OVS database connection to OVN_Southbound failed with error: 'Timeout'. Verify that the OVS and OVN services are available and that the 'ovn_nb_connection' and 'ovn_sb_connection' configuration options are correct.
2021-04-15 22:30:08.552 69189 ERROR neutron_lib.callbacks.manager Traceback (most recent call last):
2021-04-15 22:30:08.552 69189 ERROR neutron_lib.callbacks.manager File "/usr/lib/python3/dist-packages/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 67, in start_connection
2021-04-15 22:30:08.552 69189 ERROR neutron_lib.callbacks.manager self.ovsdb_connection.start()
2021-04-15 22:30:08.552 69189 ERROR neutron_lib.callbacks.manager File "/usr/lib/python3/dist-packages/ovsdbapp/backend/ovs_idl/connection.py", line 79, in start
2021-04-15 22:30:08.552 69189 ERROR neutron_lib.callbacks.manager idlutils.wait_for_change(self.idl, self.timeout)
2021-04-15 22:30:08.552 69189 ERROR neutron_lib.callbacks.manager File "/usr/lib/python3/dist-packages/ovsdbapp/backend/ovs_idl/idlutils.py", line 173, in wait_for_change
2021-04-15 22:30:08.552 69189 ERROR neutron_lib.callbacks.manager raise Exception("Timeout") # TODO(twilson) use TimeoutException?
2021-04-15 22:30:08.552 69189 ERROR neutron_lib.callbacks.manager Exception: Timeout
2021-04-15 22:30:08.552 69189 ERROR neutron_lib.callbacks.manager
2021-04-15 22:30:08.552 69189 ERROR neutron_lib.callbacks.manager During handling of the above exception, another exception occurred:
2021-04-15 22:30:08.552 69189 ERROR neutron_lib.callbacks.manager
2021-04-15 22:30:08.552 69189 ERROR neutron_lib.callbacks.manager Traceback (most recent call last):
2021-04-15 22:30:08.552 69189 ERROR neutron_lib.callbacks.manager File "/usr/lib/python3/dist-packages/neutron_lib/callbacks/manager.py", line 197, in _notify_loop
2021-04-15 22:30:08.552 69189 ERROR neutron_lib.callbacks.manager callback(resource, event, trigger, **kwargs)
2021-04-15 22:30:08.552 69189 ERROR neutron_lib.callbacks.manager File "/usr/lib/python3/dist-packages/neutron/agent/ovn/metadata/server.py", line 60, in post_fork_initialize
2021-04-15 22:30:08.552 69189 ERROR neutron_lib.callbacks.manager tables=('Port_Binding', 'Datapath_Binding')).start()
2021-04-15 22:30:08.552 69189 ERROR neutron_lib.callbacks.manager File "/usr/lib/python3/dist-packages/neutron/agent/ovn/metadata/ovsdb.py", line 57, in start
2021-04-15 22:30:08.552 69189 ERROR neutron_lib.callbacks.manager return impl_idl_ovn.OvsdbSbOvnIdl(conn)
2021-04-15 22:30:08.552 69189 ERROR neutron_lib.callbacks.manager File "/usr/lib/python3/dist-packages/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 724, in __init__
2021-04-15 22:30:08.552 69189 ERROR neutron_lib.callbacks.manager super(OvsdbSbOvnIdl, self).__init__(connection)
2021-04-15 22:30:08.552 69189 ERROR neutron_lib.callbacks.manager File "/usr/lib/python3/dist-packages/ovsdbapp/schema/ovn_southbound/impl_idl.py", line 26, in __init__
2021-04-15 22:30:08.552 69189 ERROR neutron_lib.callbacks.manager super(OvnSbApiIdlImpl, self).__init__(connection)
2021-04-15 22:30:08.552 69189 ERROR neutron_lib.callbacks.manager File "/usr/lib/python3/dist-packages/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 63, in __init__
2021-04-15 22:30:08.552 69189 ERROR neutron_lib.callbacks.manager super(OvsdbSbOvnIdl, self).__init__(connection)
2021-04-15 22:30:08.552 69189 ERROR neutron_lib.callbacks.manager File "/usr/lib/python3/dist-packages/ovsdbapp/schema/ovn_southbound/impl_idl.py", line 26, in __init__
2021-04-15 22:30:08.552 69189 ERROR neutron_lib.callbacks.manager super(OvnSbApiIdlImpl, self).__init__(connection)
2021-04-15 22:30:08.552 69189 ERROR neutron_lib.callbacks.manager File "/usr/lib/python3/dist-packages/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 63, in __init__
2021-04-15 22:30:08.552 69189 ERROR neutron_lib.callbacks.manager super(Backend, self).__init__(connection)
2021-04-15 22:30:08.552 69189 ERROR neutron_lib.callbacks.manager File "/usr/lib/python3/dist-packages/ovsdbapp/backend/ovs_idl/__init__.py", line 32, in __init__
2021-04-15 22:30:08.552 69189 ERROR neutron_lib.callbacks.manager self.start_connection(connection)
2021-04-15 22:30:08.552 69189 ERROR neutron_lib.callbacks.manager File "/usr/lib/python3/dist-packages/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 72, in start_connection
2021-04-15 22:30:08.552 69189 ERROR neutron_lib.callbacks.manager raise connection_exception
2021-04-15 22:30:08.552 69189 ERROR neutron_lib.callbacks.manager neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn.OvsdbConnectionUnavailable: OVS database connection to OVN_Southbound failed with error: 'Timeout'. Verify that the OVS and OVN services are available and that the 'ovn_nb_connection' and 'ovn_sb_connection' configuration options are correct.
2021-04-15 22:30:08.552 69189 ERROR neutron_lib.callbacks.manager
2021-04-15 22:30:08.555 69189 INFO eventlet.wsgi.server [-] (69189) wsgi starting up on http:/var/lib/neutron/metadata_proxy
* However netstat shows the TCP connection to SB IDL is established
for both the workers. So this could be some problem in data at IDL.
$ grep 10.216.241.118 sosreport-*-2021-04-27-edidnbl/sos_commands/networking/netstat_-W_-neopa
tcp 0 0 10.216.241.244:53284 10.216.241.118:6642 ESTABLISHED 0 407222779 69124/neutron-ovn-m off (0.00/0/0)
tcp 1490260 0 10.216.241.244:53282 10.216.241.118:6642 ESTABLISHED 0 407288937 69189/neutron-ovn-m off (0.00/0/0)
tcp 1490260 0 10.216.241.244:53280 10.216.241.118:6642 ESTABLISHED 0 407156544 69188/neutron-ovn-m off (0.00/0/0)
It will be good if sb_idl is not initialised, the worker thread can be
closed or neutron-ovn-metadata-agent should be stopped so that the
problem will be notified to operator at the earliest.
To manage notifications about this bug go to:
https://bugs.launchpad.net/charm-ovn-chassis/+bug/1928031/+subscriptions
More information about the Ubuntu-openstack-bugs
mailing list