[Bug 1973276] Re: OVN port loses its virtual type after port update

[Brian Murray]

Hello Gregory, or anyone else affected,

Accepted neutron into focal-proposed. The package will build now and be
available at
https://launchpad.net/ubuntu/+source/neutron/2:16.4.2-0ubuntu4 in a few
hours, and then in the -proposed repository.

Please help us by testing this new package.  See
https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how
to enable and use -proposed.  Your feedback will aid us getting this
update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug,
mentioning the version of the package you tested, what testing has been
performed on the package and change the tag from verification-needed-
focal to verification-done-focal. If it does not fix the bug for you,
please add a comment stating that, and change the tag to verification-
failed-focal. In either case, without details of your testing we will
not be able to proceed.

Further information regarding the verification process can be found at
https://wiki.ubuntu.com/QATeam/PerformingSRUVerification .  Thank you in
advance for helping!

N.B. The updated package will be released to -updates after the bug(s)
fixed by this package have been verified and the package has been in
-proposed for a minimum of 7 days.

** Changed in: neutron (Ubuntu Focal)
       Status: Triaged => Fix Committed

** Tags added: verification-needed verification-needed-focal

-- 
You received this bug notification because you are a member of Ubuntu
OpenStack, which is subscribed to Ubuntu Cloud Archive.
https://bugs.launchpad.net/bugs/1973276

Title:
  OVN port loses its virtual type after port update

Status in Ubuntu Cloud Archive:
  Fix Released
Status in Ubuntu Cloud Archive ussuri series:
  Triaged
Status in Ubuntu Cloud Archive victoria series:
  Fix Committed
Status in Ubuntu Cloud Archive wallaby series:
  Fix Released
Status in Ubuntu Cloud Archive xena series:
  Fix Released
Status in Ubuntu Cloud Archive yoga series:
  Fix Released
Status in Ubuntu Cloud Archive zed series:
  Fix Released
Status in neutron:
  Fix Released
Status in neutron package in Ubuntu:
  Fix Released
Status in neutron source package in Focal:
  Fix Committed

Bug description:
  Bug found in Octavia (master)

  Octavia creates at least 2 ports for each load balancer:
  - the VIP port, it is down, it keeps/stores the IP address of the LB
  - the VRRP port, plugged into a VM, it has the VIP address in the allowed-address list (and the VIP address is configured on the interface in the VM)

  When sending an ARP request for the VIP address, the VRRP port should
  reply with its mac-address.

  In OVN the VIP port is marked as "type: virtual".

  But when the VIP port is updated, it loses its "port: virtual" status
  and that breaks the ARP resolution (OVN replies to the ARP request by
  sending the mac-address of the VIP port - which is not used/down).

  Quick reproducer that simulates the Octavia behavior:

  ===========================

  import subprocess
  import time

  import openstack

  conn = openstack.connect(cloud="devstack-admin-demo")

  network = conn.network.find_network("public")

  sg = conn.network.find_security_group('sg')
  if not sg:
      sg = conn.network.create_security_group(name='sg')

  vip_port = conn.network.create_port(
      name="lb-vip",
      network_id=network.id,
      device_id="lb-1",
      device_owner="me",
      is_admin_state_up=False)

  vip_address = [
      fixed_ip['ip_address']
      for fixed_ip in vip_port.fixed_ips
      if '.' in fixed_ip['ip_address']][0]

  vrrp_port = conn.network.create_port(
      name="lb-vrrp",
      device_id="vrrp",
      device_owner="vm",
      network_id=network.id)
  vrrp_port = conn.network.update_port(
      vrrp_port,
      allowed_address_pairs=[
          {"ip_address": vip_address,
           "mac_address": vrrp_port.mac_address}])

  time.sleep(1)

  output = subprocess.check_output(
      f"sudo ovn-nbctl show | grep -A2 'port {vip_port.id}'",
      shell=True)
  output = output.decode('utf-8')

  if 'type: virtual' in output:
      print("Port is virtual, this is ok.")
  print(output)

  conn.network.update_port(
      vip_port,
      security_group_ids=[sg.id])

  time.sleep(1)

  output = subprocess.check_output(
      f"sudo ovn-nbctl show | grep -A2 'port {vip_port.id}'",
      shell=True)
  output = output.decode('utf-8')

  if 'type: virtual' not in output:
      print("Port is not virtual, this is an issue.")
  print(output)

  ===========================

  In my env (devstack master on c9s):
  $ python3 /mnt/host/virtual_port_issue.py
  Port is virtual, this is ok.
      port e0fe2894-e306-42d9-8c5e-6e77b77659e2 (aka lb-vip)
          type: virtual
          addresses: ["fa:16:3e:93:00:8f 172.24.4.111 2001:db8::178"]

  Port is not virtual, this is an issue.
      port e0fe2894-e306-42d9-8c5e-6e77b77659e2 (aka lb-vip)
          addresses: ["fa:16:3e:93:00:8f 172.24.4.111 2001:db8::178"]
      port 8ec36278-82b1-436b-bc5e-ea03ef22192f

  In Octavia, the "port: virtual" is _sometimes_ back after other
  updates of the ports, but in some cases the LB is unreachable.

  (and "ovn-nbctl lsp-set-type <vip-port-id> virtual" fixes the LB)

  === Ubuntu SRU Details ===

  [Impact]
  This bug causes loadbalancer vip ports to lose their "virtual" type in ovn and results in broken connectivity to amphora vms after failover. There are two patches, one that fixes new ports and one that retroactively fixes existing ones. We are backporting the former since it is clean and simple but the latter does not apply cleanly so we will defer.

  [Test Case]
    * deploy openstack ussuri or victoria with neutron + ovn and octavia
    * create a loadbalancer
    * check ovn-nbctl for the vip port and check that type is virtual
    * failover the loadbalancer
    * check ovn-nbctl for the vip port and check that type is still virtual and that lb vip is reachable

  [Where things could go wrong]
  There are not anticipated to be any regressions from this backport.

To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-archive/+bug/1973276/+subscriptions