[Bug 1584647] Re: [SRU] "Interface monitor is not active" can be observed at ovs-agent start

[Łukasz Zemczak]

I will be conditionally accepting this into xenial-proposed without
artful having the fix for LP: #1752838 (as it's blocked by the previous
neutron SRU). Normally we would wait, but seeing that it will migrate
sooner or later anyway, I'd prefer to accept this and wait with the
situation to clear out in -proposed instead.

** Changed in: neutron (Ubuntu Xenial)
       Status: Triaged => Fix Committed

** Tags added: verification-needed verification-needed-xenial

-- 
You received this bug notification because you are a member of Ubuntu
OpenStack, which is subscribed to neutron in Ubuntu.
https://bugs.launchpad.net/bugs/1584647

Title:
  [SRU] "Interface monitor is not active" can be observed at ovs-agent
  start

Status in Ubuntu Cloud Archive:
  Invalid
Status in Ubuntu Cloud Archive mitaka series:
  Triaged
Status in neutron:
  Fix Released
Status in neutron package in Ubuntu:
  Invalid
Status in neutron source package in Xenial:
  Fix Committed

Bug description:
  [Impact]

  Requesting to backport to Mitaka since we are seeing this issue in
  Mitaka clouds (and fix landed in Newton) whereby some compute nodes
  fail to have their flows added to br-tun following restart of
  openvswitch-switch.

  [Test Case]

  * Deploy Openstack Mitaka with one compute host
  * Create an instance with overlay network (gre)
  * Make a note of flows added to br-tun (ovs-vsctl dump-flows br-tun)
  * systemctl restart openvswitch-switch
  * Check that flows are re-added to br-tun (compare with previous output)
  * Ensure you do not see "Interface monitor is not active" in /var/log/neutron/neutron-openvswitch-agent

  NOTE: the root cause of this issue is that ovsdb monitor async process
  that neutron-openvswitch-agent starts takes too long to start and is
  not active by the time the rpc_loop tries to poll for updates. It is
  hard to simulate this scenario and as such it is difficult to know
  whether it has happened and resolved by this patch. Nevertheless this
  patch is small and known to have resolved the issue for newer versions
  of Openstack.

  [Regression Potential]

  I can't think how this patch could cause a regression. The only
  possible difference could be that the rpc_loop might take longer to
  update flows on ovs restart but that in itself would indicate a wider
  system issue beyond the neutron service that would not constitute a
  regression.

  ---------------

  I noticed this error message in neutron-ovs-agent log when start
  neutron-openvswitch-agent

  ERROR neutron.agent.linux.ovsdb_monitor [req-a7c7a398-a13b-490e-
  adf8-c5afb24b4b9c None None] Interface monitor is not active.

  ovs-agent will start ovsdb_monitor at [1], and first use it at [2].
  There is no guarantee that ovsdb_monitor is ready at [2]. So, I can
  see the error when start neutron-openvswitch-agent.

  We should block the start to wait for the process to be active, and
  then use it. Or else, the use of ovsdb_monitor will be meaningless.

  [1]
  https://github.com/openstack/neutron/blob/6da27a78f42db00c91a747861eafde7edc6f1fa7/neutron/agent/linux/polling.py#L35

  [2]
  https://github.com/openstack/neutron/blob/6da27a78f42db00c91a747861eafde7edc6f1fa7/neutron/plugins/ml2/drivers/openvswitch/agent/ovs_neutron_agent.py#L1994

To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-archive/+bug/1584647/+subscriptions