[Bug 1985062] Re: ovsdbapp ssl send socket error

[Alin-Gabriel Serdean]

** Patch added: "openvswitch_2.17.2-0ubuntu0.22.04.1~cloud0ubuntu1.debdiff"
   https://bugs.launchpad.net/ubuntu/+source/openvswitch/+bug/1985062/+attachment/5619595/+files/openvswitch_2.17.2-0ubuntu0.22.04.1~cloud0ubuntu1.debdiff

-- 
You received this bug notification because you are a member of Ubuntu
OpenStack, which is subscribed to openvswitch in Ubuntu.
https://bugs.launchpad.net/bugs/1985062

Title:
  ovsdbapp ssl send socket error

Status in openvswitch:
  New
Status in openvswitch package in Ubuntu:
  New
Status in openvswitch source package in Jammy:
  New
Status in openvswitch source package in Kinetic:
  New

Bug description:
  ovsdbapps are sometimes failing with the following callstack:

  2022-08-10 00:00:22.087 646221 ERROR ovsdbapp.backend.ovs_idl.connection [req-d6abd5b4-69aa-4a93-ab42-82d80e75500e - - - - -] non-zero flags not allowed in calls to send() on <class 'eventlet.green.ssl.GreenSSLSocket'>: ValueError: non-zero flags not allowed in calls to send() on <class 'eventlet.green.ssl.GreenSSLSocket'>
  2022-08-10 00:00:22.087 646221 ERROR ovsdbapp.backend.ovs_idl.connection Traceback (most recent call last):
  2022-08-10 00:00:22.087 646221 ERROR ovsdbapp.backend.ovs_idl.connection   File "/usr/lib/python3/dist-packages/ovsdbapp/backend/ovs_idl/connection.py", line 107, in run
  2022-08-10 00:00:22.087 646221 ERROR ovsdbapp.backend.ovs_idl.connection     self.idl.run()
  2022-08-10 00:00:22.087 646221 ERROR ovsdbapp.backend.ovs_idl.connection   File "/usr/lib/python3/dist-packages/ovs/db/idl.py", line 433, in run
  2022-08-10 00:00:22.087 646221 ERROR ovsdbapp.backend.ovs_idl.connection     self._session.run()
  2022-08-10 00:00:22.087 646221 ERROR ovsdbapp.backend.ovs_idl.connection   File "/usr/lib/python3/dist-packages/ovs/jsonrpc.py", line 519, in run
  2022-08-10 00:00:22.087 646221 ERROR ovsdbapp.backend.ovs_idl.connection     error = self.stream.connect()
  2022-08-10 00:00:22.087 646221 ERROR ovsdbapp.backend.ovs_idl.connection   File "/usr/lib/python3/dist-packages/ovs/stream.py", line 817, in connect
  2022-08-10 00:00:22.087 646221 ERROR ovsdbapp.backend.ovs_idl.connection     retval = super(SSLStream, self).connect()
  2022-08-10 00:00:22.087 646221 ERROR ovsdbapp.backend.ovs_idl.connection   File "/usr/lib/python3/dist-packages/ovs/stream.py", line 300, in connect
  2022-08-10 00:00:22.087 646221 ERROR ovsdbapp.backend.ovs_idl.connection     self.__scs_connecting()
  2022-08-10 00:00:22.087 646221 ERROR ovsdbapp.backend.ovs_idl.connection   File "/usr/lib/python3/dist-packages/ovs/stream.py", line 268, in __scs_connecting
  2022-08-10 00:00:22.087 646221 ERROR ovsdbapp.backend.ovs_idl.connection     retval = self.check_connection_completion(self.socket)
  2022-08-10 00:00:22.087 646221 ERROR ovsdbapp.backend.ovs_idl.connection   File "/usr/lib/python3/dist-packages/ovs/stream.py", line 777, in check_connection_completion
  2022-08-10 00:00:22.087 646221 ERROR ovsdbapp.backend.ovs_idl.connection     return Stream.check_connection_completion(sock)
  2022-08-10 00:00:22.087 646221 ERROR ovsdbapp.backend.ovs_idl.connection   File "/usr/lib/python3/dist-packages/ovs/stream.py", line 137, in check_connection_completion
  2022-08-10 00:00:22.087 646221 ERROR ovsdbapp.backend.ovs_idl.connection     return ovs.socket_util.check_connection_completion(sock)
  2022-08-10 00:00:22.087 646221 ERROR ovsdbapp.backend.ovs_idl.connection   File "/usr/lib/python3/dist-packages/ovs/socket_util.py", line 181, in check_connection_completion
  2022-08-10 00:00:22.087 646221 ERROR ovsdbapp.backend.ovs_idl.connection     sock.send("\0".encode(), socket.MSG_DONTWAIT)
  2022-08-10 00:00:22.087 646221 ERROR ovsdbapp.backend.ovs_idl.connection   File "/usr/lib/python3/dist-packages/eventlet/green/ssl.py", line 193, in send
  2022-08-10 00:00:22.087 646221 ERROR ovsdbapp.backend.ovs_idl.connection     return self._call_trampolining(
  2022-08-10 00:00:22.087 646221 ERROR ovsdbapp.backend.ovs_idl.connection   File "/usr/lib/python3/dist-packages/eventlet/green/ssl.py", line 157, in _call_trampolining
  2022-08-10 00:00:22.087 646221 ERROR ovsdbapp.backend.ovs_idl.connection     return func(*a, **kw)
  2022-08-10 00:00:22.087 646221 ERROR ovsdbapp.backend.ovs_idl.connection   File "/usr/lib/python3.8/ssl.py", line 1170, in send
  2022-08-10 00:00:22.087 646221 ERROR ovsdbapp.backend.ovs_idl.connection     raise ValueError(
  2022-08-10 00:00:22.087 646221 ERROR ovsdbapp.backend.ovs_idl.connection ValueError: non-zero flags not allowed in calls to send() on <class 'eventlet.green.ssl.GreenSSLSocket'>
  2022-08-10 00:00:22.087 646221 ERROR ovsdbapp.backend.ovs_idl.connection 
  2022-08-10 00:00:22.088 646221 ERROR ovsdbapp.backend.ovs_idl.connection [req-d6abd5b4-69aa-4a93-ab42-82d80e75500e - - - - -] non-zero flags not allowed in calls to send() on <class 'eventlet.green.ssl.GreenSSLSocket'>: ValueError: non-zero flags not allowed in calls to send() on <class 'eventlet.green.ssl.GreenSSLSocket'>
  2022-08-10 00:00:22.088 646221 ERROR ovsdbapp.backend.ovs_idl.connection Traceback (most recent call last):
  2022-08-10 00:00:22.088 646221 ERROR ovsdbapp.backend.ovs_idl.connection   File "/usr/lib/python3/dist-packages/ovsdbapp/backend/ovs_idl/connection.py", line 107, in run
  2022-08-10 00:00:22.088 646221 ERROR ovsdbapp.backend.ovs_idl.connection     self.idl.run()
  2022-08-10 00:00:22.088 646221 ERROR ovsdbapp.backend.ovs_idl.connection   File "/usr/lib/python3/dist-packages/ovs/db/idl.py", line 433, in run
  2022-08-10 00:00:22.088 646221 ERROR ovsdbapp.backend.ovs_idl.connection     self._session.run()
  2022-08-10 00:00:22.088 646221 ERROR ovsdbapp.backend.ovs_idl.connection   File "/usr/lib/python3/dist-packages/ovs/jsonrpc.py", line 519, in run
  2022-08-10 00:00:22.088 646221 ERROR ovsdbapp.backend.ovs_idl.connection     error = self.stream.connect()
  2022-08-10 00:00:22.088 646221 ERROR ovsdbapp.backend.ovs_idl.connection   File "/usr/lib/python3/dist-packages/ovs/stream.py", line 817, in connect
  2022-08-10 00:00:22.088 646221 ERROR ovsdbapp.backend.ovs_idl.connection     retval = super(SSLStream, self).connect()
  2022-08-10 00:00:22.088 646221 ERROR ovsdbapp.backend.ovs_idl.connection   File "/usr/lib/python3/dist-packages/ovs/stream.py", line 300, in connect
  2022-08-10 00:00:22.088 646221 ERROR ovsdbapp.backend.ovs_idl.connection     self.__scs_connecting()
  2022-08-10 00:00:22.088 646221 ERROR ovsdbapp.backend.ovs_idl.connection   File "/usr/lib/python3/dist-packages/ovs/stream.py", line 268, in __scs_connecting
  2022-08-10 00:00:22.088 646221 ERROR ovsdbapp.backend.ovs_idl.connection     retval = self.check_connection_completion(self.socket)
  2022-08-10 00:00:22.088 646221 ERROR ovsdbapp.backend.ovs_idl.connection   File "/usr/lib/python3/dist-packages/ovs/stream.py", line 777, in check_connection_completion
  2022-08-10 00:00:22.088 646221 ERROR ovsdbapp.backend.ovs_idl.connection     return Stream.check_connection_completion(sock)
  2022-08-10 00:00:22.088 646221 ERROR ovsdbapp.backend.ovs_idl.connection   File "/usr/lib/python3/dist-packages/ovs/stream.py", line 137, in check_connection_completion
  2022-08-10 00:00:22.088 646221 ERROR ovsdbapp.backend.ovs_idl.connection     return ovs.socket_util.check_connection_completion(sock)
  2022-08-10 00:00:22.088 646221 ERROR ovsdbapp.backend.ovs_idl.connection   File "/usr/lib/python3/dist-packages/ovs/socket_util.py", line 181, in check_connection_completion
  2022-08-10 00:00:22.088 646221 ERROR ovsdbapp.backend.ovs_idl.connection     sock.send("\0".encode(), socket.MSG_DONTWAIT)
  2022-08-10 00:00:22.088 646221 ERROR ovsdbapp.backend.ovs_idl.connection   File "/usr/lib/python3/dist-packages/eventlet/green/ssl.py", line 193, in send
  2022-08-10 00:00:22.088 646221 ERROR ovsdbapp.backend.ovs_idl.connection     return self._call_trampolining(
  2022-08-10 00:00:22.088 646221 ERROR ovsdbapp.backend.ovs_idl.connection   File "/usr/lib/python3/dist-packages/eventlet/green/ssl.py", line 157, in _call_trampolining
  2022-08-10 00:00:22.088 646221 ERROR ovsdbapp.backend.ovs_idl.connection     return func(*a, **kw)
  2022-08-10 00:00:22.088 646221 ERROR ovsdbapp.backend.ovs_idl.connection   File "/usr/lib/python3.8/ssl.py", line 1170, in send
  2022-08-10 00:00:22.088 646221 ERROR ovsdbapp.backend.ovs_idl.connection     raise ValueError(
  2022-08-10 00:00:22.088 646221 ERROR ovsdbapp.backend.ovs_idl.connection ValueError: non-zero flags not allowed in calls to send() on <class 'eventlet.green.ssl.GreenSSLSocket'>
  2022-08-10 00:00:22.088 646221 ERROR ovsdbapp.backend.ovs_idl.connection 
  2022-08-10 00:00:22.088 646221 ERROR ovsdbapp.backend.ovs_idl.connection During handling of the above exception, another exception occurred:
  2022-08-10 00:00:22.088 646221 ERROR ovsdbapp.backend.ovs_idl.connection 
  2022-08-10 00:00:22.088 646221 ERROR ovsdbapp.backend.ovs_idl.connection Traceback (most recent call last):
  2022-08-10 00:00:22.088 646221 ERROR ovsdbapp.backend.ovs_idl.connection   File "/usr/lib/python3/dist-packages/ovsdbapp/backend/ovs_idl/connection.py", line 116, in run
  2022-08-10 00:00:22.088 646221 ERROR ovsdbapp.backend.ovs_idl.connection     idlutils.wait_for_change(self.idl, self.timeout)
  2022-08-10 00:00:22.088 646221 ERROR ovsdbapp.backend.ovs_idl.connection   File "/usr/lib/python3/dist-packages/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 51, in wait_for_change
  2022-08-10 00:00:22.088 646221 ERROR ovsdbapp.backend.ovs_idl.connection     while idl_.change_seqno == seqno and not idl_.run():
  2022-08-10 00:00:22.088 646221 ERROR ovsdbapp.backend.ovs_idl.connection   File "/usr/lib/python3/dist-packages/ovs/db/idl.py", line 433, in run
  2022-08-10 00:00:22.088 646221 ERROR ovsdbapp.backend.ovs_idl.connection     self._session.run()
  2022-08-10 00:00:22.088 646221 ERROR ovsdbapp.backend.ovs_idl.connection   File "/usr/lib/python3/dist-packages/ovs/jsonrpc.py", line 519, in run
  2022-08-10 00:00:22.088 646221 ERROR ovsdbapp.backend.ovs_idl.connection     error = self.stream.connect()
  2022-08-10 00:00:22.088 646221 ERROR ovsdbapp.backend.ovs_idl.connection   File "/usr/lib/python3/dist-packages/ovs/stream.py", line 817, in connect
  2022-08-10 00:00:22.088 646221 ERROR ovsdbapp.backend.ovs_idl.connection     retval = super(SSLStream, self).connect()
  2022-08-10 00:00:22.088 646221 ERROR ovsdbapp.backend.ovs_idl.connection   File "/usr/lib/python3/dist-packages/ovs/stream.py", line 300, in connect
  2022-08-10 00:00:22.088 646221 ERROR ovsdbapp.backend.ovs_idl.connection     self.__scs_connecting()
  2022-08-10 00:00:22.088 646221 ERROR ovsdbapp.backend.ovs_idl.connection   File "/usr/lib/python3/dist-packages/ovs/stream.py", line 268, in __scs_connecting
  2022-08-10 00:00:22.088 646221 ERROR ovsdbapp.backend.ovs_idl.connection     retval = self.check_connection_completion(self.socket)
  2022-08-10 00:00:22.088 646221 ERROR ovsdbapp.backend.ovs_idl.connection   File "/usr/lib/python3/dist-packages/ovs/stream.py", line 777, in check_connection_completion
  2022-08-10 00:00:22.088 646221 ERROR ovsdbapp.backend.ovs_idl.connection     return Stream.check_connection_completion(sock)
  2022-08-10 00:00:22.088 646221 ERROR ovsdbapp.backend.ovs_idl.connection   File "/usr/lib/python3/dist-packages/ovs/stream.py", line 137, in check_connection_completion
  2022-08-10 00:00:22.088 646221 ERROR ovsdbapp.backend.ovs_idl.connection     return ovs.socket_util.check_connection_completion(sock)
  2022-08-10 00:00:22.088 646221 ERROR ovsdbapp.backend.ovs_idl.connection   File "/usr/lib/python3/dist-packages/ovs/socket_util.py", line 181, in check_connection_completion
  2022-08-10 00:00:22.088 646221 ERROR ovsdbapp.backend.ovs_idl.connection     sock.send("\0".encode(), socket.MSG_DONTWAIT)
  2022-08-10 00:00:22.088 646221 ERROR ovsdbapp.backend.ovs_idl.connection   File "/usr/lib/python3/dist-packages/eventlet/green/ssl.py", line 193, in send
  2022-08-10 00:00:22.088 646221 ERROR ovsdbapp.backend.ovs_idl.connection     return self._call_trampolining(
  2022-08-10 00:00:22.088 646221 ERROR ovsdbapp.backend.ovs_idl.connection   File "/usr/lib/python3/dist-packages/eventlet/green/ssl.py", line 157, in _call_trampolining
  2022-08-10 00:00:22.088 646221 ERROR ovsdbapp.backend.ovs_idl.connection     return func(*a, **kw)
  2022-08-10 00:00:22.088 646221 ERROR ovsdbapp.backend.ovs_idl.connection   File "/usr/lib/python3.8/ssl.py", line 1170, in send
  2022-08-10 00:00:22.088 646221 ERROR ovsdbapp.backend.ovs_idl.connection     raise ValueError(
  2022-08-10 00:00:22.088 646221 ERROR ovsdbapp.backend.ovs_idl.connection ValueError: non-zero flags not allowed in calls to send() on <class 'eventlet.green.ssl.GreenSSLSocket'>
  2022-08-10 00:00:22.088 646221 ERROR ovsdbapp.backend.ovs_idl.connection 
  2022-08-10 00:00:22.089 646221 INFO ovsdbapp.backend.ovs_idl.connection [req-d6abd5b4-69aa-4a93-ab42-82d80e75500e - - - - -] Trying to recover, sleeping 60 seconds

  The following relevant OVS packages are installed

  root at ps5-rb1-n1:~# dpkg -l | grep openvswitch
  ii  openvswitch-common                    2.17.2-0ubuntu0                                      amd64        Open vSwitch common components
  ii  openvswitch-switch                    2.17.2-0ubuntu0                                      amd64        Open vSwitch switch implementations
  ii  python3-openvswitch                   2.17.2-0ubuntu0                                      amd64        Python 3 bindings for Open vSwitch

  This should be fixed by the following patch proposed but not yet commited upstream:
  https://patchwork.ozlabs.org/project/openvswitch/patch/20220808173242.955238-1-mtomaska@redhat.com/

To manage notifications about this bug go to:
https://bugs.launchpad.net/openvswitch/+bug/1985062/+subscriptions