[Bug 1894843] Re: [dvr_snat] Router update deletes rfp interface from qrouter even when VM port is present on this host

Hemanth Nakkina 1894843 at bugs.launchpad.net
Fri Apr 16 09:26:58 UTC 2021


Verified on bionic-train and working fine.

Output of ping VM FIP (launched on non-snat node) during disable/enable
router

$ ping 10.5.150.244
PING 10.5.150.244 (10.5.150.244) 56(84) bytes of data.
64 bytes from 10.5.150.244: icmp_seq=1 ttl=62 time=393 ms
64 bytes from 10.5.150.244: icmp_seq=2 ttl=62 time=0.955 ms
>From 10.5.151.36 icmp_seq=8 Redirect Host(New nexthop: 244.150.5.10)
>From 10.5.151.36 icmp_seq=9 Redirect Host(New nexthop: 244.150.5.10)
>From 10.5.151.36 icmp_seq=10 Redirect Host(New nexthop: 244.150.5.10)
>From 10.5.151.36 icmp_seq=17 Destination Host Unreachable
>From 10.5.151.36 icmp_seq=20 Destination Host Unreachable
>From 10.5.151.36 icmp_seq=21 Destination Host Unreachable
>From 10.5.151.36 icmp_seq=22 Destination Host Unreachable
64 bytes from 10.5.150.244: icmp_seq=27 ttl=62 time=1.10 ms
64 bytes from 10.5.150.244: icmp_seq=28 ttl=62 time=0.750 ms
64 bytes from 10.5.150.244: icmp_seq=29 ttl=62 time=0.840 ms


** Tags removed: verification-train-needed
** Tags added: verification-train-done

-- 
You received this bug notification because you are a member of Ubuntu
OpenStack, which is subscribed to Ubuntu Cloud Archive.
https://bugs.launchpad.net/bugs/1894843

Title:
  [dvr_snat] Router update deletes rfp interface from qrouter even when
  VM port is present on this host

Status in Ubuntu Cloud Archive:
  Fix Committed
Status in Ubuntu Cloud Archive queens series:
  Triaged
Status in Ubuntu Cloud Archive rocky series:
  Fix Released
Status in Ubuntu Cloud Archive stein series:
  Fix Released
Status in Ubuntu Cloud Archive train series:
  Fix Committed
Status in Ubuntu Cloud Archive ussuri series:
  Fix Released
Status in Ubuntu Cloud Archive victoria series:
  Fix Released
Status in neutron:
  Fix Released
Status in neutron package in Ubuntu:
  Fix Released
Status in neutron source package in Bionic:
  Triaged
Status in neutron source package in Focal:
  Fix Released
Status in neutron source package in Groovy:
  Fix Released
Status in neutron source package in Hirsute:
  Fix Released

Bug description:
  [Impact]
  When neutron schedules snat namespaces it sometimes deletes the rfp interface from qrouter namespaces which breaks external network (fip) connectivity. The fix prevents this from happening.

  [Test Case]
   * deploy Openstack (Ussuri or above) with dvr_snat enabled in compute hosts.
   * ensure min. 2 compute hosts
   * create one ext network and one private network
   * add private subnet to router and ext as gateway
   * check which compute has the snat ns (ip netns| grep snat)
   * create a vm on each compute host
   * check that qrouter ns on both computes has rfp interface
   * ip netns| grep qrouter; ip netns exec <ns> ip a s| grep rfp
   * disable and re-enable router
   * openstack router set --disable <router>;  openstack router set --enable <router>
   * check again
   * ip netns| grep qrouter; ip netns exec <ns> ip a s| grep rfp

  [Where problems could occur]
  This patch is in fact restoring expected behaviour and is not expected to
  introduce any new regressions.

  -------------------------------------------------------------------------

  Hello,

  In the case of dvr_snat l3 agents are deployed on hypervisors there
  can be race condition. The agent creates snat namespaces on each
  scheduled host and removes them at second step. At this second step
  agent removes the rfp interface from qrouter even when there is VM
  with floating IP on the host.

  When VM is deployed at the time of second step we can lost external
  access to VMs floating IP. The issue can be reproduced by hand:

  1. Create tenant network and router with external gateway
  2. Create VM with floating ip
  3. Ensure that VM on the hypervisor without snat-* namespace
  4. Set the router to disabled state (openstack router set --disable <router>)
  5. Set the router to enabled state (openstack router set --enabled <router>)
  6. The external access to VMs FIP have lost because L3 agent creates the qrouter namespace without rfp interface.

  Environment:

  1. Neutron with ML2 OVS plugin.
  2. L3 agents in dvr_snat mode on each hypervisor
  3. openstack-neutron-common-15.1.1-0.20200611111910.7d97420.el8ost.noarch

To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-archive/+bug/1894843/+subscriptions



More information about the Ubuntu-openstack-bugs mailing list