[Bug 2007919] Re: [SRU] Stable point releases for python-ovsdbapp

Thu Sep 14 13:11:19 UTC 2023

Thanks for the review Robie. I appreciate your thorough reviews and
keeping Ubuntu stable!

(Note: The following reply is taken from my reply to a related thread:
https://lists.ubuntu.com/archives/ubuntu-
release/2023-September/005765.html)

For the behavioral change [1], I think there is minimal regression potential as the change is limited to an exception path where sleeps and reconnects were determined to be unnecessary. I'd have appreciated it if they would have provided a bug reference for the commit to provide more context, however with that said, it seems they are making a code improvement here for performance reasons and they found it to be useful enough to backport to stable branches.
[1] https://opendev.org/openstack/ovsdbapp/commit/ab3e0cb0d0865417efbf103f44954573a5ba92ac

It is certainly not ideal to have missing unit tests. It's important to
consider that we perform testing above and beyond the unit tests that
are run as part of our package builds. The regression testing that we
have in place for OpenStack is run for all stable releases. For that we
deploy a full OpenStack cloud and then run tempest integration tests
that exercise all of the packages and the cloud solution as a whole.

I think we have a solid history of limiting regression potential for
OpenStack users in our stable releases and that remains very important
to us.

-- 
You received this bug notification because you are a member of Ubuntu
OpenStack, which is subscribed to python-ovsdbapp in Ubuntu.
https://bugs.launchpad.net/bugs/2007919

Title:
  [SRU] Stable point releases for python-ovsdbapp

Status in OpenStack Neutron API Charm:
  Invalid
Status in Ubuntu Cloud Archive:
  Invalid
Status in Ubuntu Cloud Archive wallaby series:
  Fix Committed
Status in Ubuntu Cloud Archive xena series:
  Fix Committed
Status in Ubuntu Cloud Archive yoga series:
  Triaged
Status in Ubuntu Cloud Archive zed series:
  Fix Committed
Status in python-ovsdbapp package in Ubuntu:
  Invalid
Status in python-ovsdbapp source package in Jammy:
  New

Bug description:
  [Impact]
  This release sports mostly bug-fixes and we would like to make
  sure all of our supported customers have access to these
  improvements. The update contains the following package updates:

  python-ovsdbapp 2.1.1 (zed)
  python-ovsdbapp 1.15.3 (jammy/yoga)
  python-ovsdbapp 1.12.3 (xena)
  python-ovsdbapp 1.9.4 (wallaby)

  Delta between 2.1.0 and 2.1.1 (zed):

   - [97e738d] Fix TRY_AGAIN handling
   - [bbdc14a] Don't force_reconnect() on unhandled Idl exception
   - [c9e9473] Fix logging exception while handling transaction exception
   - [d90358c] Update TOX_CONSTRAINTS_FILE for stable/zed
   - [b58a340] Update .gitreview for stable/zed

  Delta between 1.15.1 and 1.15.3 (jammy/yoga):

   - [ab3e0cb] Don't force_reconnect() on unhandled Idl exception
   - [92cbba4] Fix logging exception while handling transaction exception
   - [315c809] Fix TRY_AGAIN handling
   - [a69430a] Handle OVS 2.17 change to Idl.tables
   - [de473ce] Fix broken behavior after recent OVS update
   - [10539ed] github no longer accepts git:// connections
   - [7735d5c] Update TOX_CONSTRAINTS_FILE for stable/yoga
   - [253afd4] Update .gitreview for stable/yoga

  Delta between 1.12.0 and 1.12.3 (xena):

   - [8202d53] Don't force_reconnect() on unhandled Idl exception
   - [1b04058] Fix logging exception while handling transaction exception
   - [fc62ae2] Fix TRY_AGAIN handling
   - [4d4402c] Update QoS register in "QoSAddCommand" if exists
   - [62ee947] Add cooperative_yield() to OvsdbIdl
   - [ab571f4] Handle OVS 2.17 change to Idl.tables
   - [a7a1d5c] Fix broken behavior after recent OVS update
   - [a73fbf1] github no longer accepts git:// connections
   - [3a0cd11] Log what exception caused abort of the transaction
   - [8c09568] Update TOX_CONSTRAINTS_FILE for stable/xena
   - [19a4364] Update .gitreview for stable/xena
   - [49ab130] tools: run functional tests against ovn/main branch

  Delta between 1.9.0 and 1.9.4 (wallaby):

   - [65d02f0] Don't force_reconnect() on unhandled Idl exception
   - [87dcab2] Fix logging exception while handling transaction exception
   - [dd7e332] Fix TRY_AGAIN handling
   - [f804411] Update QoS register in "QoSAddCommand" if exists
   - [e1a0d7c] Add cooperative_yield() to OvsdbIdl
   - [c6e2c43] Handle OVS 2.17 change to Idl.tables
   - [9990509] Fix broken behavior after recent OVS update
   - [fb5feee] github no longer accepts git:// connections
   - [fa4a747] Log what exception caused abort of the transaction
   - [eaa6406] tools: run functional tests against ovn/main branch
   - [8304251] Actually close the connection in Connection.stop()
   - [66918f9] Don't spam retries 100s of times a second
   - [d0cc492] Add an active wait in the "Backend.lookup"
   - [007c0f6] Fix docs job
   - [f6a19a3] Update TOX_CONSTRAINTS_FILE for stable/wallaby
   - [875be60] Update .gitreview for stable/wallaby

  [Test Case]

  The microrelease testing documentation at
  https://wiki.ubuntu.com/StableReleaseUpdates requests 4 points of
  testing, shown below, along with my comments on the coverage that we
  currently have in place:

  * a reliable and credible test suite for assuring the quality of every
  commit or release,

  coreycb: Upstream has unit tests and they are run during the package
  build.

  * the tests are covering both functionality and API/ABI stability,

  coreycb: This would be covered by unit tests as well as our planned
  functional testing described at [1].

  * the tests run during package build to cover all architectures,

  coreycb: Unit tests are run during package builds. Since this is a
  pure python package it is only built on amd64.

  * the package has an autopkgtest to run the tests in an Ubuntu
  environment against the actual binary packages"

  coreycb: The autopkgtests only test python imports. Since the
  regression testing described in [1] will be exercising the actual
  binary package functionality, I would like to see if we can get
  approval based on that being an adequate alternative.

  [1]
  The following SRU process was followed:
  https://wiki.ubuntu.com/OpenStackUpdates

  In order to avoid regression of existing consumers, the OpenStack team
  will run their continuous integration test against the packages that
  are in -proposed. A successful run of all available tests will be
  required before the proposed packages can be let into -updates.

  The OpenStack team will be in charge of attaching the output summary
  of the executed tests. The OpenStack team members will not mark
  ‘verification-done’ until this has happened.

  [Regression Potential]
  In order to mitigate the regression potential, the results of the
  aforementioned tests are attached to this bug.

  [Discussion]
  I would like to provide these new stable releases via microrelease SRU updates: https://wiki.ubuntu.com/StableReleaseUpdates#New_upstream_microreleases

  The recommended releases that upstream tests with are listed at:
  https://github.com/openstack/requirements/blob/stable/yoga/upper-constraints.txt
  https://github.com/openstack/requirements/blob/stable/xena/upper-constraints.txt
  https://github.com/openstack/requirements/blob/stable/wallaby/upper-constraints.txt

  [Original Bug Description]

  Subject: ovsdbapp: AttributeError: 'TableSchema' object has no
  attribute 'condition_state'

  After upgrade from victoria to wallaby neutron-api units firing
  errors. Exception is related to
  https://bugs.launchpad.net/ovsdbapp/+bug/1972989, which is fixed in
  ovsdbapp 1.9.3

  https://releases.openstack.org/wallaby/ states that ovsdbapp should be
  1.9.4 (latest for walalby)

  Installed in the unit is:

  root at juju-4c2163-3-lxd-14:~# python3 -m pip list | grep ovsdbapp
  ovsdbapp                1.9.0

  Also:
  root at juju-4c2163-3-lxd-14:~# apt-cache policy python3-ovsdbapp
  python3-ovsdbapp:
    Installed: 1.9.0-0ubuntu1~cloud0
    Candidate: 1.9.0-0ubuntu1~cloud0
    Version table:
   *** 1.9.0-0ubuntu1~cloud0 500
          500 http://ubuntu-cloud.archive.canonical.com/ubuntu focal-updates/wallaby/main amd64 Packages
          100 /var/lib/dpkg/status
       1.1.0-0ubuntu2 500
          500 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 Packages
       1.1.0-0ubuntu1 500
          500 http://archive.ubuntu.com/ubuntu focal/main amd64 Packages

  So it looks like the charm doesn't upgrade ovsdbapp during openstack
  upgrade, because a newer version with the above fix doesn't exist in
  the UCA:

   python3-ovsdbapp | 1.9.0-0ubuntu1~cloud0  | wallaby           | focal-updates   | all
   python3-ovsdbapp | 1.9.0-0ubuntu1~cloud0  | wallaby-proposed  | focal-proposed  | all

  Full traceback:

  2023-02-21 07:39:19.639 431391 ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn   File "/usr/lib/python3/dist-packages/ovsdbapp/backend/ovs_idl/connection.py", line 81, in start
  2023-02-21 07:39:19.639 431391 ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn     idlutils.wait_for_change(self.idl, self.timeout)
  2023-02-21 07:39:19.639 431391 ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn   File "/usr/lib/python3/dist-packages/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 52, in wait_for_change
  2023-02-21 07:39:19.639 431391 ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn     while idl_.change_seqno == seqno and not idl_.run():
  2023-02-21 07:39:19.639 431391 ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn   File "/usr/lib/python3/dist-packages/ovs/db/idl.py", line 541, in run
  2023-02-21 07:39:19.639 431391 ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn     self.__send_monitor_request()
  2023-02-21 07:39:19.639 431391 ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn   File "/usr/lib/python3/dist-packages/ovs/db/idl.py", line 850, in __send_monitor_request
  2023-02-21 07:39:19.639 431391 ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn     not ConditionState.is_true(table.condition_state.acked)):
  2023-02-21 07:39:19.639 431391 ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn AttributeError: 'TableSchema' object has no attribute 'condition_state'

  Worked around by:
  juju run -a neutron-api 'sudo python3 -m pip install ovsdbapp==1.9.4'
  juju run -a neutron-api 'sudo service neutron-* restart'

To manage notifications about this bug go to:
https://bugs.launchpad.net/charm-neutron-api/+bug/2007919/+subscriptions