[Bug 1861457] Re: pyroute2 0.5.2 doesn't support neutron-common 14.0.4

James Page james.page at ubuntu.com
Fri Jan 31 12:18:48 UTC 2020


Working on reproducing this issue so we can identify exactly what the
issue is.

FTR I'm not adverse to bumping to pyroute2 0.5.3 in the stein UCA if
that resolves the issue (as that the minimum version identified in
neutron anyway).

** Changed in: neutron (Ubuntu)
     Assignee: (unassigned) => James Page (james-page)

-- 
You received this bug notification because you are a member of Ubuntu
OpenStack, which is subscribed to neutron in Ubuntu.
https://bugs.launchpad.net/bugs/1861457

Title:
  pyroute2 0.5.2 doesn't support neutron-common 14.0.4

Status in OpenStack neutron-gateway charm:
  Invalid
Status in neutron package in Ubuntu:
  New

Bug description:
  On a Bionic-Stein clouds we recently updated all packages and
  encountered issues creating routers namespaces due to some changes in
  the data structure expected from pyroute2.

  For Bionic/Stein, the latest neutron-common package is from 14.0.4 and
  the latest pyroute2 is 0.5.2.

  We were able to resolve the issue by forcing an update of pyroute to 0.5.6 from the Eoan repository on n-gw units and cycling the neutron-openvswitch-agent and neutron-l3-agent services.
  Fixes will need to be back ported and released to Bionic Stein repo.

  
  2020-01-30 20:00:49.261 1874349 DEBUG oslo_concurrency.lockutils [-] Releasing lock "iptables-qrouter-5241d017-9d8a-44b4-912f-d36703bbc259" lock /usr/lib/python3/dist-packages/oslo_concurrency/lockutils.py:281
  2020-01-30 20:00:49.262 1874349 ERROR neutron.agent.l3.router_info [-] 255: KeyError: 255
  2020-01-30 20:00:49.262 1874349 ERROR neutron.agent.l3.router_info Traceback (most recent call last):
  2020-01-30 20:00:49.262 1874349 ERROR neutron.agent.l3.router_info   File "/usr/lib/python3/dist-packages/neutron/common/utils.py", line 158, in call
  2020-01-30 20:00:49.262 1874349 ERROR neutron.agent.l3.router_info     return func(*args, **kwargs)
  2020-01-30 20:00:49.262 1874349 ERROR neutron.agent.l3.router_info   File "/usr/lib/python3/dist-packages/neutron/agent/l3/router_info.py", line 1188, in process
  2020-01-30 20:00:49.262 1874349 ERROR neutron.agent.l3.router_info     self.process_external()
  2020-01-30 20:00:49.262 1874349 ERROR neutron.agent.l3.router_info   File "/usr/lib/python3/dist-packages/neutron/agent/l3/router_info.py", line 946, in process_external
  2020-01-30 20:00:49.262 1874349 ERROR neutron.agent.l3.router_info     self._process_external_gateway(ex_gw_port)
  2020-01-30 20:00:49.262 1874349 ERROR neutron.agent.l3.router_info   File "/usr/lib/python3/dist-packages/neutron/agent/l3/router_info.py", line 828, in _process_external_gateway
  2020-01-30 20:00:49.262 1874349 ERROR neutron.agent.l3.router_info     self.external_gateway_added(ex_gw_port, interface_name)
  2020-01-30 20:00:49.262 1874349 ERROR neutron.agent.l3.router_info   File "/usr/lib/python3/dist-packages/neutron/agent/l3/ha_router.py", line 434, in external_gateway_added
  2020-01-30 20:00:49.262 1874349 ERROR neutron.agent.l3.router_info     self._plug_external_gateway(ex_gw_port, interface_name, self.ns_name)
  2020-01-30 20:00:49.262 1874349 ERROR neutron.agent.l3.router_info   File "/usr/lib/python3/dist-packages/neutron/agent/l3/router_info.py", line 670, in _plug_external_gateway
  2020-01-30 20:00:49.262 1874349 ERROR neutron.agent.l3.router_info     mtu=ex_gw_port.get('mtu'))
  2020-01-30 20:00:49.262 1874349 ERROR neutron.agent.l3.router_info   File "/usr/lib/python3/dist-packages/neutron/agent/linux/interface.py", line 267, in plug
  2020-01-30 20:00:49.262 1874349 ERROR neutron.agent.l3.router_info     bridge, namespace, prefix, mtu)
  2020-01-30 20:00:49.262 1874349 ERROR neutron.agent.l3.router_info   File "/usr/lib/python3/dist-packages/neutron/agent/linux/interface.py", line 355, in plug_new
  2020-01-30 20:00:49.262 1874349 ERROR neutron.agent.l3.router_info     namespace2=namespace)
  2020-01-30 20:00:49.262 1874349 ERROR neutron.agent.l3.router_info   File "/usr/lib/python3/dist-packages/neutron/agent/linux/ip_lib.py", line 213, in add_veth
  2020-01-30 20:00:49.262 1874349 ERROR neutron.agent.l3.router_info     name1, self.namespace, 'veth', peer=peer)
  2020-01-30 20:00:49.262 1874349 ERROR neutron.agent.l3.router_info   File "/usr/lib/python3/dist-packages/neutron/privileged/agent/linux/ip_lib.py", line 50, in sync_inner
  2020-01-30 20:00:49.262 1874349 ERROR neutron.agent.l3.router_info     return input_func(*args, **kwargs)
  2020-01-30 20:00:49.262 1874349 ERROR neutron.agent.l3.router_info   File "/usr/lib/python3/dist-packages/oslo_privsep/priv_context.py", line 241, in _wrap
  2020-01-30 20:00:49.262 1874349 ERROR neutron.agent.l3.router_info     return self.channel.remote_call(name, args, kwargs)
  2020-01-30 20:00:49.262 1874349 ERROR neutron.agent.l3.router_info   File "/usr/lib/python3/dist-packages/oslo_privsep/daemon.py", line 203, in remote_call
  2020-01-30 20:00:49.262 1874349 ERROR neutron.agent.l3.router_info     raise exc_type(*result[2])
  2020-01-30 20:00:49.262 1874349 ERROR neutron.agent.l3.router_info KeyError: 255
  2020-01-30 20:00:49.262 1874349 ERROR neutron.agent.l3.router_info
  2020-01-30 20:00:49.263 1874349 ERROR neutron.agent.l3.agent [-] Failed to process compatible router: 5241d017-9d8a-44b4-912f-d36703bbc259: KeyError: 255
  2020-01-30 20:00:49.263 1874349 ERROR neutron.agent.l3.agent Traceback (most recent call last):
  2020-01-30 20:00:49.263 1874349 ERROR neutron.agent.l3.agent   File "/usr/lib/python3/dist-packages/neutron/agent/l3/agent.py", line 723, in _process_routers_if_compatible
  2020-01-30 20:00:49.263 1874349 ERROR neutron.agent.l3.agent     self._process_router_if_compatible(router)
  2020-01-30 20:00:49.263 1874349 ERROR neutron.agent.l3.agent   File "/usr/lib/python3/dist-packages/neutron/agent/l3/agent.py", line 567, in _process_router_if_compatible
  2020-01-30 20:00:49.263 1874349 ERROR neutron.agent.l3.agent     self._process_added_router(router)
  2020-01-30 20:00:49.263 1874349 ERROR neutron.agent.l3.agent   File "/usr/lib/python3/dist-packages/neutron/agent/l3/agent.py", line 575, in _process_added_router
  2020-01-30 20:00:49.263 1874349 ERROR neutron.agent.l3.agent     ri.process()
  2020-01-30 20:00:49.263 1874349 ERROR neutron.agent.l3.agent   File "/usr/lib/python3/dist-packages/neutron/agent/l3/ha_router.py", line 485, in process
  2020-01-30 20:00:49.263 1874349 ERROR neutron.agent.l3.agent     super(HaRouter, self).process()
  2020-01-30 20:00:49.263 1874349 ERROR neutron.agent.l3.agent   File "/usr/lib/python3/dist-packages/neutron/common/utils.py", line 161, in call
  2020-01-30 20:00:49.263 1874349 ERROR neutron.agent.l3.agent     self.logger(e)
  2020-01-30 20:00:49.263 1874349 ERROR neutron.agent.l3.agent   File "/usr/lib/python3/dist-packages/oslo_utils/excutils.py", line 220, in __exit__
  2020-01-30 20:00:49.263 1874349 ERROR neutron.agent.l3.agent     self.force_reraise()
  2020-01-30 20:00:49.263 1874349 ERROR neutron.agent.l3.agent   File "/usr/lib/python3/dist-packages/oslo_utils/excutils.py", line 196, in force_reraise
  2020-01-30 20:00:49.263 1874349 ERROR neutron.agent.l3.agent     six.reraise(self.type_, self.value, self.tb)
  2020-01-30 20:00:49.263 1874349 ERROR neutron.agent.l3.agent   File "/usr/lib/python3/dist-packages/six.py", line 693, in reraise
  2020-01-30 20:00:49.263 1874349 ERROR neutron.agent.l3.agent     raise value
  2020-01-30 20:00:49.263 1874349 ERROR neutron.agent.l3.agent   File "/usr/lib/python3/dist-packages/neutron/common/utils.py", line 158, in call
  2020-01-30 20:00:49.263 1874349 ERROR neutron.agent.l3.agent     return func(*args, **kwargs)
  2020-01-30 20:00:49.263 1874349 ERROR neutron.agent.l3.agent   File "/usr/lib/python3/dist-packages/neutron/agent/l3/router_info.py", line 1188, in process
  2020-01-30 20:00:49.263 1874349 ERROR neutron.agent.l3.agent     self.process_external()
  2020-01-30 20:00:49.263 1874349 ERROR neutron.agent.l3.agent   File "/usr/lib/python3/dist-packages/neutron/agent/l3/router_info.py", line 946, in process_external
  2020-01-30 20:00:49.263 1874349 ERROR neutron.agent.l3.agent     self._process_external_gateway(ex_gw_port)
  2020-01-30 20:00:49.263 1874349 ERROR neutron.agent.l3.agent   File "/usr/lib/python3/dist-packages/neutron/agent/l3/router_info.py", line 828, in _process_external_gateway
  2020-01-30 20:00:49.263 1874349 ERROR neutron.agent.l3.agent     self.external_gateway_added(ex_gw_port, interface_name)
  2020-01-30 20:00:49.263 1874349 ERROR neutron.agent.l3.agent   File "/usr/lib/python3/dist-packages/neutron/agent/l3/ha_router.py", line 434, in external_gateway_added
  2020-01-30 20:00:49.263 1874349 ERROR neutron.agent.l3.agent     self._plug_external_gateway(ex_gw_port, interface_name, self.ns_name)
  2020-01-30 20:00:49.263 1874349 ERROR neutron.agent.l3.agent   File "/usr/lib/python3/dist-packages/neutron/agent/l3/router_info.py", line 670, in _plug_external_gateway
  2020-01-30 20:00:49.263 1874349 ERROR neutron.agent.l3.agent     mtu=ex_gw_port.get('mtu'))
  2020-01-30 20:00:49.263 1874349 ERROR neutron.agent.l3.agent   File "/usr/lib/python3/dist-packages/neutron/agent/linux/interface.py", line 267, in plug
  2020-01-30 20:00:49.263 1874349 ERROR neutron.agent.l3.agent     bridge, namespace, prefix, mtu)
  2020-01-30 20:00:49.263 1874349 ERROR neutron.agent.l3.agent   File "/usr/lib/python3/dist-packages/neutron/agent/linux/interface.py", line 355, in plug_new
  2020-01-30 20:00:49.263 1874349 ERROR neutron.agent.l3.agent     namespace2=namespace)
  2020-01-30 20:00:49.263 1874349 ERROR neutron.agent.l3.agent   File "/usr/lib/python3/dist-packages/neutron/agent/linux/ip_lib.py", line 213, in add_veth
  2020-01-30 20:00:49.263 1874349 ERROR neutron.agent.l3.agent     name1, self.namespace, 'veth', peer=peer)
  2020-01-30 20:00:49.263 1874349 ERROR neutron.agent.l3.agent   File "/usr/lib/python3/dist-packages/neutron/privileged/agent/linux/ip_lib.py", line 50, in sync_inner
  2020-01-30 20:00:49.263 1874349 ERROR neutron.agent.l3.agent     return input_func(*args, **kwargs)
  2020-01-30 20:00:49.263 1874349 ERROR neutron.agent.l3.agent   File "/usr/lib/python3/dist-packages/oslo_privsep/priv_context.py", line 241, in _wrap
  2020-01-30 20:00:49.263 1874349 ERROR neutron.agent.l3.agent     return self.channel.remote_call(name, args, kwargs)
  2020-01-30 20:00:49.263 1874349 ERROR neutron.agent.l3.agent   File "/usr/lib/python3/dist-packages/oslo_privsep/daemon.py", line 203, in remote_call
  2020-01-30 20:00:49.263 1874349 ERROR neutron.agent.l3.agent     raise exc_type(*result[2])
  2020-01-30 20:00:49.263 1874349 ERROR neutron.agent.l3.agent KeyError: 255

To manage notifications about this bug go to:
https://bugs.launchpad.net/charm-neutron-gateway/+bug/1861457/+subscriptions



More information about the Ubuntu-openstack-bugs mailing list