[Bug 1752838] Re: [SRU] neutron-ovs-cleanup runs earlier than ovsdb-server when there are many port

[Seyeong Kim]

Hello,

for xenial-queens

I followed below step to verify this. ( slightly different )

1. deploy openstack xenial-queens
2. create networks routers etc on horizon
3. create instance on horizon
4. login to host
5. adding dummy ports via ( https://paste.ubuntu.com/p/X35H85XWPh/ )
6. reboot and check if order is ok ( not ok here, at this time, systemd status failed )
7. upgrade pkg
8. reboot and check if order is ok ( ok here, clean )
9. regression check ( checking ovs-vsctl output if the port for #3 instance is exist ) on host
- service openvswitch-switch restart
- service neutron-openvswitch-agent restart
- check if port are still there

Thanks

dpkg -l | grep  neutron
ii  neutron-common                       2:12.0.0-0ubuntu2~cloud0                    all          Neutron is a virtual network service for Openstack - common
ii  neutron-openvswitch-agent            2:12.0.0-0ubuntu2~cloud0                    all          Neutron is a virtual network service for Openstack - Open vSwitch plugin agent
ii  neutron-plugin-openvswitch-agent     2:12.0.0-0ubuntu2~cloud0                    all          Transitional package for neutron-openvswitch-agent

** Tags removed: verification-queens-needed
** Tags added: verification-queens-done

-- 
You received this bug notification because you are a member of Ubuntu
OpenStack, which is subscribed to neutron in Ubuntu.
https://bugs.launchpad.net/bugs/1752838

Title:
  [SRU] neutron-ovs-cleanup runs earlier than ovsdb-server when there
  are many port

Status in Ubuntu Cloud Archive:
  Fix Committed
Status in Ubuntu Cloud Archive newton series:
  Fix Committed
Status in Ubuntu Cloud Archive ocata series:
  Fix Committed
Status in Ubuntu Cloud Archive pike series:
  Triaged
Status in Ubuntu Cloud Archive queens series:
  Fix Committed
Status in neutron package in Ubuntu:
  Fix Released
Status in neutron source package in Xenial:
  Triaged
Status in neutron source package in Artful:
  Triaged
Status in neutron source package in Bionic:
  Fix Released

Bug description:
  [Impact]

  In code, Basically all ubuntu version could be affected.
  I'm trying to do test them

  neutron-ovs-cleanup service runs earlier than ovsdb-server sometimes.

  then, neutron-ovs-cleanup service's status remains failed
  with below error msg

  "ProcessExecutionError: Exit code: 1; Stdin: ; Stdout: ; Stderr: ovs-
  vsctl: unix:/var/run/openvswitch/db.sock: database connection failed
  (No such file or directory)"

  that is fine for booting.

  but.

  If someday upgrade pkgs related to neutron, it runs neutron-ovs-
  cleanup again because it is not in active status.

  eventually, neutron-ovs-cleanup deletes all ports on running compute
  nodes.

  related logs here https://paste.ubuntu.com/p/QkkwP2vPDv/

  [Test Case]

  1. deploy openstack env
  2. run below scripts to add many ports
  #!/bin/bash

  juju ssh nova-compute/0 '
  sudo modprobe dummy
  for i in {1..1000}
  do
    echo $i
    sudo ip link add dummy$i type dummy;sudo ip link set name t-test$i dev dummy$i;sudo ovs-vsctl add-port br-int t-test$i
  done
  '
  3. systemctl list-units --failed
  - check if there is failed neutron-ovs-cleanup service

  [Regression]
  Changes dependency on debian/neutron-openvswitch-agent.neutron-ovs-cleanup.service.in
  need to reboot if this patch is affected. If this symptom is not affected your env. no need to reboot.

To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-archive/+bug/1752838/+subscriptions