[Bug 1859649] Re: neutron 2:14.0.3-0ubuntu1~cloud0 and 2:14.0.0-0ubuntu1.1~cloud0 not compatible

[Junien Fridrick]

The other in neutron-l3-agent (still on neutron-gateway), but this is
probably due to the upgrade restarting the daemons :

neutron-l3-agent.log:2020-01-14 16:22:11.298 456968 ERROR pyroute2.netns.nslink [-] forced shutdown procedure, clean up netns manually
neutron-l3-agent.log:2020-01-14 16:22:11.322 456743 ERROR neutron.agent.l3.router_info [-] [Errno 9] Bad file descriptor: OSError: [Errno 9] Bad file descriptor
neutron-l3-agent.log:2020-01-14 16:22:11.322 456743 ERROR neutron.agent.l3.router_info Traceback (most recent call last):
neutron-l3-agent.log:2020-01-14 16:22:11.322 456743 ERROR neutron.agent.l3.router_info   File "/usr/lib/python3/dist-packages/neutron/common/utils.py", line 158, in call
neutron-l3-agent.log:2020-01-14 16:22:11.322 456743 ERROR neutron.agent.l3.router_info     return func(*args, **kwargs)
neutron-l3-agent.log:2020-01-14 16:22:11.322 456743 ERROR neutron.agent.l3.router_info   File "/usr/lib/python3/dist-packages/neutron/agent/l3/router_info.py", line 1186, in process
neutron-l3-agent.log:2020-01-14 16:22:11.322 456743 ERROR neutron.agent.l3.router_info     self._process_internal_ports()
neutron-l3-agent.log:2020-01-14 16:22:11.322 456743 ERROR neutron.agent.l3.router_info   File "/usr/lib/python3/dist-packages/neutron/agent/l3/router_info.py", line 584, in _process_internal_ports
neutron-l3-agent.log:2020-01-14 16:22:11.322 456743 ERROR neutron.agent.l3.router_info     self.internal_network_added(p)
neutron-l3-agent.log:2020-01-14 16:22:11.322 456743 ERROR neutron.agent.l3.router_info   File "/usr/lib/python3/dist-packages/neutron/agent/l3/router_info.py", line 486, in internal_network_added
neutron-l3-agent.log:2020-01-14 16:22:11.322 456743 ERROR neutron.agent.l3.router_info     mtu=port.get('mtu'))
neutron-l3-agent.log:2020-01-14 16:22:11.322 456743 ERROR neutron.agent.l3.router_info   File "/usr/lib/python3/dist-packages/neutron/agent/l3/router_info.py", line 461, in _internal_network_added
neutron-l3-agent.log:2020-01-14 16:22:11.322 456743 ERROR neutron.agent.l3.router_info     prefix=prefix, mtu=mtu)
neutron-l3-agent.log:2020-01-14 16:22:11.322 456743 ERROR neutron.agent.l3.router_info   File "/usr/lib/python3/dist-packages/neutron/agent/linux/interface.py", line 265, in plug
neutron-l3-agent.log:2020-01-14 16:22:11.322 456743 ERROR neutron.agent.l3.router_info     namespace=namespace):
neutron-l3-agent.log:2020-01-14 16:22:11.322 456743 ERROR neutron.agent.l3.router_info   File "/usr/lib/python3/dist-packages/neutron/agent/linux/ip_lib.py", line 818, in device_exists
neutron-l3-agent.log:2020-01-14 16:22:11.322 456743 ERROR neutron.agent.l3.router_info     return IPDevice(device_name, namespace=namespace).exists()
neutron-l3-agent.log:2020-01-14 16:22:11.322 456743 ERROR neutron.agent.l3.router_info   File "/usr/lib/python3/dist-packages/neutron/agent/linux/ip_lib.py", line 318, in exists
neutron-l3-agent.log:2020-01-14 16:22:11.322 456743 ERROR neutron.agent.l3.router_info     return privileged.interface_exists(self.name, self.namespace)
neutron-l3-agent.log:2020-01-14 16:22:11.322 456743 ERROR neutron.agent.l3.router_info   File "/usr/lib/python3/dist-packages/neutron/privileged/agent/linux/ip_lib.py", line 50, in sync_inner
neutron-l3-agent.log:2020-01-14 16:22:11.322 456743 ERROR neutron.agent.l3.router_info     return input_func(*args, **kwargs)
neutron-l3-agent.log:2020-01-14 16:22:11.322 456743 ERROR neutron.agent.l3.router_info   File "/usr/lib/python3/dist-packages/oslo_privsep/priv_context.py", line 241, in _wrap
neutron-l3-agent.log:2020-01-14 16:22:11.322 456743 ERROR neutron.agent.l3.router_info     return self.channel.remote_call(name, args, kwargs)
neutron-l3-agent.log:2020-01-14 16:22:11.322 456743 ERROR neutron.agent.l3.router_info   File "/usr/lib/python3/dist-packages/oslo_privsep/daemon.py", line 203, in remote_call
neutron-l3-agent.log:2020-01-14 16:22:11.322 456743 ERROR neutron.agent.l3.router_info     raise exc_type(*result[2])
neutron-l3-agent.log:2020-01-14 16:22:11.322 456743 ERROR neutron.agent.l3.router_info OSError: [Errno 9] Bad file descriptor
neutron-l3-agent.log:2020-01-14 16:22:11.322 456743 ERROR neutron.agent.l3.router_info
neutron-l3-agent.log:2020-01-14 16:22:11.327 456743 ERROR neutron.agent.l3.agent [-] Failed to process compatible router: 209673dc-a96b-4367-8b48-08bdf8f5ad5e: OSError: [Errno 9] Bad file descriptor
neutron-l3-agent.log:2020-01-14 16:22:11.327 456743 ERROR neutron.agent.l3.agent Traceback (most recent call last):
neutron-l3-agent.log:2020-01-14 16:22:11.327 456743 ERROR neutron.agent.l3.agent   File "/usr/lib/python3/dist-packages/neutron/agent/l3/agent.py", line 723, in _process_routers_if_compatible
neutron-l3-agent.log:2020-01-14 16:22:11.327 456743 ERROR neutron.agent.l3.agent     self._process_router_if_compatible(router)
neutron-l3-agent.log:2020-01-14 16:22:11.327 456743 ERROR neutron.agent.l3.agent   File "/usr/lib/python3/dist-packages/neutron/agent/l3/agent.py", line 567, in _process_router_if_compatible
neutron-l3-agent.log:2020-01-14 16:22:11.327 456743 ERROR neutron.agent.l3.agent     self._process_added_router(router)
neutron-l3-agent.log:2020-01-14 16:22:11.327 456743 ERROR neutron.agent.l3.agent   File "/usr/lib/python3/dist-packages/neutron/agent/l3/agent.py", line 575, in _process_added_router
neutron-l3-agent.log:2020-01-14 16:22:11.327 456743 ERROR neutron.agent.l3.agent     ri.process()
neutron-l3-agent.log:2020-01-14 16:22:11.327 456743 ERROR neutron.agent.l3.agent   File "/usr/lib/python3/dist-packages/neutron/common/utils.py", line 161, in call
neutron-l3-agent.log:2020-01-14 16:22:11.327 456743 ERROR neutron.agent.l3.agent     self.logger(e)
neutron-l3-agent.log:2020-01-14 16:22:11.327 456743 ERROR neutron.agent.l3.agent   File "/usr/lib/python3/dist-packages/oslo_utils/excutils.py", line 220, in __exit__
neutron-l3-agent.log:2020-01-14 16:22:11.327 456743 ERROR neutron.agent.l3.agent     self.force_reraise()
neutron-l3-agent.log:2020-01-14 16:22:11.327 456743 ERROR neutron.agent.l3.agent   File "/usr/lib/python3/dist-packages/oslo_utils/excutils.py", line 196, in force_reraise
neutron-l3-agent.log:2020-01-14 16:22:11.327 456743 ERROR neutron.agent.l3.agent     six.reraise(self.type_, self.value, self.tb)
neutron-l3-agent.log:2020-01-14 16:22:11.327 456743 ERROR neutron.agent.l3.agent   File "/usr/lib/python3/dist-packages/six.py", line 693, in reraise
neutron-l3-agent.log:2020-01-14 16:22:11.327 456743 ERROR neutron.agent.l3.agent     raise value
neutron-l3-agent.log:2020-01-14 16:22:11.327 456743 ERROR neutron.agent.l3.agent   File "/usr/lib/python3/dist-packages/neutron/common/utils.py", line 158, in call
neutron-l3-agent.log:2020-01-14 16:22:11.327 456743 ERROR neutron.agent.l3.agent     return func(*args, **kwargs)
neutron-l3-agent.log:2020-01-14 16:22:11.327 456743 ERROR neutron.agent.l3.agent   File "/usr/lib/python3/dist-packages/neutron/agent/l3/router_info.py", line 1186, in process
neutron-l3-agent.log:2020-01-14 16:22:11.327 456743 ERROR neutron.agent.l3.agent     self._process_internal_ports()
neutron-l3-agent.log:2020-01-14 16:22:11.327 456743 ERROR neutron.agent.l3.agent   File "/usr/lib/python3/dist-packages/neutron/agent/l3/router_info.py", line 584, in _process_internal_ports
neutron-l3-agent.log:2020-01-14 16:22:11.327 456743 ERROR neutron.agent.l3.agent     self.internal_network_added(p)
neutron-l3-agent.log:2020-01-14 16:22:11.327 456743 ERROR neutron.agent.l3.agent   File "/usr/lib/python3/dist-packages/neutron/agent/l3/router_info.py", line 486, in internal_network_added
neutron-l3-agent.log:2020-01-14 16:22:11.327 456743 ERROR neutron.agent.l3.agent     mtu=port.get('mtu'))
neutron-l3-agent.log:2020-01-14 16:22:11.327 456743 ERROR neutron.agent.l3.agent   File "/usr/lib/python3/dist-packages/neutron/agent/l3/router_info.py", line 461, in _internal_network_added
neutron-l3-agent.log:2020-01-14 16:22:11.327 456743 ERROR neutron.agent.l3.agent     prefix=prefix, mtu=mtu)
neutron-l3-agent.log:2020-01-14 16:22:11.327 456743 ERROR neutron.agent.l3.agent   File "/usr/lib/python3/dist-packages/neutron/agent/linux/interface.py", line 265, in plug
neutron-l3-agent.log:2020-01-14 16:22:11.327 456743 ERROR neutron.agent.l3.agent     namespace=namespace):
neutron-l3-agent.log:2020-01-14 16:22:11.327 456743 ERROR neutron.agent.l3.agent   File "/usr/lib/python3/dist-packages/neutron/agent/linux/ip_lib.py", line 818, in device_exists
neutron-l3-agent.log:2020-01-14 16:22:11.327 456743 ERROR neutron.agent.l3.agent     return IPDevice(device_name, namespace=namespace).exists()
neutron-l3-agent.log:2020-01-14 16:22:11.327 456743 ERROR neutron.agent.l3.agent   File "/usr/lib/python3/dist-packages/neutron/agent/linux/ip_lib.py", line 318, in exists
neutron-l3-agent.log:2020-01-14 16:22:11.327 456743 ERROR neutron.agent.l3.agent     return privileged.interface_exists(self.name, self.namespace)
neutron-l3-agent.log:2020-01-14 16:22:11.327 456743 ERROR neutron.agent.l3.agent   File "/usr/lib/python3/dist-packages/neutron/privileged/agent/linux/ip_lib.py", line 50, in sync_inner
neutron-l3-agent.log:2020-01-14 16:22:11.327 456743 ERROR neutron.agent.l3.agent     return input_func(*args, **kwargs)
neutron-l3-agent.log:2020-01-14 16:22:11.327 456743 ERROR neutron.agent.l3.agent   File "/usr/lib/python3/dist-packages/oslo_privsep/priv_context.py", line 241, in _wrap
neutron-l3-agent.log:2020-01-14 16:22:11.327 456743 ERROR neutron.agent.l3.agent     return self.channel.remote_call(name, args, kwargs)
neutron-l3-agent.log:2020-01-14 16:22:11.327 456743 ERROR neutron.agent.l3.agent   File "/usr/lib/python3/dist-packages/oslo_privsep/daemon.py", line 203, in remote_call
neutron-l3-agent.log:2020-01-14 16:22:11.327 456743 ERROR neutron.agent.l3.agent     raise exc_type(*result[2])
neutron-l3-agent.log:2020-01-14 16:22:11.327 456743 ERROR neutron.agent.l3.agent OSError: [Errno 9] Bad file descriptor
neutron-l3-agent.log:2020-01-14 16:22:11.327 456743 ERROR neutron.agent.l3.agent

-- 
You received this bug notification because you are a member of Ubuntu
OpenStack, which is subscribed to neutron in Ubuntu.
https://bugs.launchpad.net/bugs/1859649

Title:
  neutron 2:14.0.3-0ubuntu1~cloud0 and 2:14.0.0-0ubuntu1.1~cloud0 not
  compatible

Status in neutron package in Ubuntu:
  New

Bug description:
  Hi,

  Not entirely sure why, but if a cloud has services running these 2
  versions of neutron (2:14.0.3-0ubuntu1~cloud0 and
  2:14.0.0-0ubuntu1.1~cloud0), networking is basically broken until
  everything is running 2:14.0.3-0ubuntu1~cloud0.

  Which causes networking disruption when not all nodes are upgraded at
  the same time.

  Thanks

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/neutron/+bug/1859649/+subscriptions