[Bug 1927868] Re: vRouter not working after update to 16.3.1
Jared Baker
1927868 at bugs.launchpad.net
Wed Jun 16 13:29:36 UTC 2021
keepalived: 1:1.3.9-1ubuntu0.18.04.2
from: http://archive.ubuntu.com/ubuntu bionic-updates/main
neutron-l3-agent.log filled with the following log lines for every single router in our cluster:
2021-06-10 19:59:53.602 328191 ERROR neutron.agent.l3.ha_router [-] Gateway interface for router 9f276b0e-8581-4312-8780-ff4503c3df07 was not set up; router will not work properly
We have 3 controllers and use l3 HA and usually have 1 l3 agent per
controller, 2 standby, 1 active.
We rolled back to 16.0.0 (only because we could not find 16.3.0
anywhere) and the issue stopped. OP was able to roll back to 16.3.0 and
said the issue went away so I think we should focus on 16.3.1+
Before upgrading to Ussuri we did briefly upgrade to the latest Train
packages, give me some time to see if I can see if those error messages
started when we were briefly on latest Train packages. I need to do some
cross-referencing for that.
--
You received this bug notification because you are a member of Ubuntu
OpenStack, which is subscribed to neutron in Ubuntu.
https://bugs.launchpad.net/bugs/1927868
Title:
vRouter not working after update to 16.3.1
Status in neutron:
New
Status in neutron package in Ubuntu:
Confirmed
Bug description:
We run a juju managed Openstack Ussuri on Bionic. After updating
neutron packages from 16.3.0 to 16.3.1 all virtual routers stopped
working. It seems that most (not all) namespaces are created but have
only the lo interface and sometime the ha-XYZ interface in DOWN state.
The underlying tap interfaces are also in down.
neutron-l3-agent has many logs similar to the following:
2021-05-08 15:01:45.286 39411 ERROR neutron.agent.l3.ha_router [-] Gateway interface for router 02945b59-639b-41be-8237-3b7933b4e32d was not set up; router will not work properly
and journal logs report at around the same time
May 08 15:01:40 lar1615.srv-louros.grnet.gr neutron-keepalived-state-change[18596]: 2021-05-08 15:01:40.765 18596 INFO neutron.agent.linux.ip_lib [-] Failed sending gratuitous ARP to 62.62.62.62 on qg-5a6efe8c-6b in namespace qrouter-02945b59-639b-41be-8237-3b7933b4e32d: Exit code: 2; Stdin: ; Stdout: Interface "qg-5a6efe8c-6b" is down
May 08 15:01:40 lar1615.srv-louros.grnet.gr neutron-keepalived-state-change[18596]: 2021-05-08 15:01:40.767 18596 INFO neutron.agent.linux.ip_lib [-] Interface qg-5a6efe8c-6b or address 62.62.62.62 in namespace qrouter-02945b59-639b-41be-8237-3b7933b4e32d was deleted concurrently
The neutron packages installed are:
ii neutron-common 2:16.3.1-0ubuntu1~cloud0 all Neutron is a virtual network service for Openstack - common
ii neutron-dhcp-agent 2:16.3.1-0ubuntu1~cloud0 all Neutron is a virtual network service for Openstack - DHCP agent
ii neutron-l3-agent 2:16.3.1-0ubuntu1~cloud0 all Neutron is a virtual network service for Openstack - l3 agent
ii neutron-metadata-agent 2:16.3.1-0ubuntu1~cloud0 all Neutron is a virtual network service for Openstack - metadata agent
ii neutron-metering-agent 2:16.3.1-0ubuntu1~cloud0 all Neutron is a virtual network service for Openstack - metering agent
ii neutron-openvswitch-agent 2:16.3.1-0ubuntu1~cloud0 all Neutron is a virtual network service for Openstack - Open vSwitch plugin agent
ii python3-neutron 2:16.3.1-0ubuntu1~cloud0 all Neutron is a virtual network service for Openstack - Python library
ii python3-neutron-lib 2.3.0-0ubuntu1~cloud0 all Neutron shared routines and utilities - Python 3.x
ii python3-neutronclient 1:7.1.1-0ubuntu1~cloud0 all client API library for Neutron - Python 3.x
Downgrading to 16.3.0 resolves the issues.
To manage notifications about this bug go to:
https://bugs.launchpad.net/neutron/+bug/1927868/+subscriptions
More information about the Ubuntu-openstack-bugs
mailing list