[Bug 1891673] [NEW] qrouter ns ip rules not deleted when fip removed from vm

[Launchpad Bug Tracker]

You have been subscribed to a public bug by Ubuntu Foundations Team Bug Bot (crichton):

[Impact]

neutron-l3-agent restart causes partial loss of fip information such
that fip removal from vm results in ip rules left behind which breaks
external network access for that vm.

[Test Case]

* deploy openstack with dvr enabled
* create distributed router, network etc
* create a vm and attach a floating ip
* go to compute host on which vm is running and restart neutron-l3-agent
* tail -f /var/log/neutron/neutron-l3-agent.log until it settles
* remove fip from vm
* run https://gist.github.com/dosaboy/eca8dcd4560f68d856f465ca8382c58b on that compute node
* should return with "nothing to do"

[Regression Potential]
none expected

[Other Info]
patched neutron l3 agent will reload info for *used* floating ips when restarted BUT if there are ip rules left behind from fips removed prior to using a pathed neutron then manual cleanup is still required and for that you can use https://gist.github.com/dosaboy/eca8dcd4560f68d856f465ca8382c58b.
 
--------------------------------------------------------------------------

With Bionic Stein using dvr_snat if I add a floating ip to a vm then
remove the floating ip, the corresponding ip rules in the associated
qrouter ns local to the instance are not deleted which results in no
longer being able to reach the external network because packets are
still sent to the fip namespace (via rfp-/fpr-) e.g. in my compute host
running a vm whose address is 192.168.21.28 for which i have removed the
fip I still see:

# ip netns exec qrouter-5e45608f-33d4-41bf-b3ba-915adf612e65 ip rule list
0:      from all lookup local
32765:  from 192.168.21.28 lookup 16
32766:  from all lookup main
32767:  from all lookup default
3232240897:     from 192.168.21.1/24 lookup 3232240897
3232241231:     from 192.168.22.79/24 lookup 3232241231

And table 16 leads to:

# ip netns exec qrouter-5e45608f-33d4-41bf-b3ba-915adf612e65 ip route show table 16
default via 169.254.109.249 dev rfp-5e45608f-3

Which results in the instance no longer being able to reach the external
network (packets are never sent to the snat- ns in my case).

The workaround is to delete that ip rule but neutron should be taking
care of this. Looks like the culprit is in
neutron/agent/l3/dvr_local_router.py:floating_ip_removed_dist

Note that the NAT rules were successfully removed from iptables so looks
like it is just this bit that is left behind.

** Affects: cloud-archive
     Importance: Undecided
         Status: New

** Affects: cloud-archive/queens
     Importance: Undecided
         Status: In Progress

** Affects: cloud-archive/rocky
     Importance: Undecided
         Status: In Progress

** Affects: cloud-archive/stein
     Importance: Undecided
         Status: In Progress

** Affects: cloud-archive/train
     Importance: Undecided
         Status: In Progress

** Affects: cloud-archive/ussuri
     Importance: Undecided
         Status: In Progress

** Affects: cloud-archive/victoria
     Importance: Undecided
         Status: New

** Affects: neutron
     Importance: High
     Assignee: Edward Hope-Morley (hopem)
         Status: Fix Released

** Affects: neutron (Ubuntu)
     Importance: Undecided
         Status: New

** Affects: neutron (Ubuntu Bionic)
     Importance: Undecided
         Status: New

** Affects: neutron (Ubuntu Focal)
     Importance: Undecided
         Status: New

** Affects: neutron (Ubuntu Groovy)
     Importance: Undecided
         Status: New


** Tags: l3-dvr-backlog patch sts sts-sru-needed
-- 
qrouter ns ip rules not deleted when fip removed from vm
https://bugs.launchpad.net/bugs/1891673
You received this bug notification because you are a member of Ubuntu Sponsors Team, which is subscribed to the bug report.