[Bug 1906280] Please test proposed package
Corey Bryant
1906280 at bugs.launchpad.net
Wed Jan 6 15:04:06 UTC 2021
Hello Michael, or anyone else affected,
Accepted openvswitch into stein-proposed. The package will build now and
be available in the Ubuntu Cloud Archive in a few hours, and then in the
-proposed repository.
Please help us by testing this new package. To enable the -proposed
repository:
sudo add-apt-repository cloud-archive:stein-proposed
sudo apt-get update
Your feedback will aid us getting this update out to other Ubuntu users.
If this package fixes the bug for you, please add a comment to this bug,
mentioning the version of the package you tested, and change the tag
from verification-stein-needed to verification-stein-done. If it does
not fix the bug for you, please add a comment stating that, and change
the tag to verification-stein-failed. In either case, details of your
testing will help us make a better decision.
Further information regarding the verification process can be found at
https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in
advance!
** Changed in: cloud-archive/stein
Status: Triaged => Fix Committed
** Tags added: verification-stein-needed
--
You received this bug notification because you are a member of Ubuntu
OpenStack, which is subscribed to openvswitch in Ubuntu.
https://bugs.launchpad.net/bugs/1906280
Title:
[SRU] Add support for disabling mlockall() calls in ovs-vswitchd
Status in OpenStack neutron-openvswitch charm:
In Progress
Status in charm-ovn-chassis:
In Progress
Status in Ubuntu Cloud Archive:
Invalid
Status in Ubuntu Cloud Archive queens series:
Triaged
Status in Ubuntu Cloud Archive stein series:
Fix Committed
Status in Ubuntu Cloud Archive train series:
Fix Committed
Status in Ubuntu Cloud Archive ussuri series:
Fix Committed
Status in openvswitch package in Ubuntu:
Fix Released
Status in openvswitch source package in Bionic:
Triaged
Status in openvswitch source package in Focal:
Fix Committed
Status in openvswitch source package in Groovy:
Fix Committed
Status in openvswitch source package in Hirsute:
Fix Released
Bug description:
[Impact]
Recent changes to systemd rlimit are resulting in memory exhaustion
with ovs-vswitchd's use of mlockall(). mlockall() can be disabled via
/etc/defaults/openvswitch-vswitch, however there is currently a bug in
the shipped ovs-vswitchd systemd unit file that prevents it. The
package will be fixed in this SRU. Additionally the neutron-
openvswitch charm will be updated to enable disabling of mlockall()
use in ovs-vswitchd via a config option.
More details on the above summary can be found in the following comments:
https://bugs.launchpad.net/charm-neutron-openvswitch/+bug/1906280/comments/16
https://bugs.launchpad.net/charm-neutron-openvswitch/+bug/1906280/comments/19
==== Original bug details ===
Original bug title:
Charm stuck waiting for ovsdb 'no key "ovn-remote" in Open_vSwitch
record'
Original bug details:
As seen during this Focal Ussuri test run: https://solutions.qa.canonical.com/testruns/testRun/5f7ad510-f57e-40ce-beb7-5f39800fa5f0
Crashdump here: https://oil-jenkins.canonical.com/artifacts/5f7ad510-f57e-40ce-beb7-5f39800fa5f0/generated/generated/openstack/juju-crashdump-openstack-2020-11-28-03.40.36.tar.gz
Full history of occurrences can be found here:
https://solutions.qa.canonical.com/bugs/bugs/bug/1906280
Octavia's ovn-chassis units are stuck waiting:
octavia/0 blocked idle 1/lxd/8 10.244.8.170 9876/tcp Awaiting leader to create required resources
hacluster-octavia/1 active idle 10.244.8.170 Unit is ready and clustered
logrotated/63 active idle 10.244.8.170 Unit is ready.
octavia-ovn-chassis/1 waiting executing 10.244.8.170 'ovsdb' incomplete
public-policy-routing/45 active idle 10.244.8.170 Unit is ready
When the db is reporting healthy:
ovn-central/0* active idle 1/lxd/9 10.246.64.225 6641/tcp,6642/tcp Unit is ready (leader: ovnnb_db, ovnsb_db)
logrotated/19 active idle 10.246.64.225 Unit is ready.
ovn-central/1 active idle 3/lxd/9 10.246.64.250 6641/tcp,6642/tcp Unit is ready (northd: active)
logrotated/27 active idle 10.246.64.250 Unit is ready.
ovn-central/2 active idle 5/lxd/9 10.246.65.21 6641/tcp,6642/tcp Unit is ready
logrotated/52 active idle 10.246.65.21 Unit is ready.
Warning in the juju unit logs indicates that the charm is blocking on
a missing key in the ovsdb:
2020-11-27 23:36:57 INFO juju-log ovsdb:195: Invoking reactive handler: hooks/relations/ovsdb-subordinate/provides.py:97:joined:ovsdb-subordinate
2020-11-27 23:36:57 DEBUG jujuc server.go:211 running hook tool "relation-get"
2020-11-27 23:36:57 WARNING ovsdb-relation-changed ovs-vsctl: no key "ovn-remote" in Open_vSwitch record "." column external_ids
2020-11-27 23:36:57 DEBUG jujuc server.go:211 running hook tool "juju-log"
2020-11-27 23:36:57 INFO juju-log ovsdb:195: Invoking reactive handler: hooks/relations/ovsdb/requires.py:34:joined:ovsdb
==============================
[Test Case]
Note: Bionic requires additional testing due to pairing with other SRUS.
The easiest way to test this is to deploy openstack with the neutron-
openvswitch charm, using the new charm updates. Once deployed, edit
/usr/share/openvswitch/scripts/ovs-ctl with an echo to show what
MLOCKALL is set to. Then toggle the charm config option [1] and look
at journalctl -xe to find the echo output, which should correspond to
the mlockall setting.
[1]
juju config neutron-openvswitch disable-mlockall=true
juju config neutron-openvswitch disable-mlockall=false
[Regression Potential]
There's potential that this will break users who have come to depend on the incorrect EnvironmentFile setting and environment variable in the systemd unit file for ovs-vswitchd. If that is the case they must be running with modified systemd unit files anyway so it is probably a moot point.
[Discussion]
== Groovy ==
Update (16-12-2020): I chatted briefly with Christian and it sounds like the ltmain-whole-archive.diff may be optional, so I've dropped it from this upload. There are now 2 openvswitch's in the groovy unapproved queue. Please reject the upload from 15-12-2020 and consider accepting the upload from 16-12-2020.
I have a query out to James and Christian about an undocumented commit that is getting picked up in the groovy upload. It is committed to the ubuntu/groovy branch of the package Vcs. See debian/ltmain-whole-archive.diff and debian/rules in the upload debdiff at http://launchpadlibrarian.net/511453613/openvswitch_2.13.1-0ubuntu1_2.13.1-0ubuntu1.1.diff.gz
== Bionic ==
The bionic upload is paired with the following SRUs which will also require verification:
https://bugs.launchpad.net/bugs/1823295
https://bugs.launchpad.net/bugs/1881077
== Package details ==
New package versions are in progress and can be found at:
hirsute: https://launchpad.net/ubuntu/+source/openvswitch/2.14.0-0ubuntu2
groovy: https://launchpad.net/ubuntu/groovy/+queue?queue_state=1&queue_text=openvswitch
focal: https://launchpad.net/ubuntu/focal/+queue?queue_state=1&queue_text=openvswitch
train: https://launchpad.net/~ubuntu-cloud-archive/+archive/ubuntu/train-staging/+packages?field.name_filter=openvswitch&field.status_filter=published&field.series_filter=
stein: https://launchpad.net/~ubuntu-cloud-archive/+archive/ubuntu/stein-staging/+packages?field.name_filter=openvswitch&field.status_filter=published&field.series_filter=
bionic: https://launchpad.net/ubuntu/bionic/+queue?queue_state=1&queue_text=openvswitch
== Charm update ==
https://review.opendev.org/c/openstack/charm-neutron-openvswitch/+/767212
To manage notifications about this bug go to:
https://bugs.launchpad.net/charm-neutron-openvswitch/+bug/1906280/+subscriptions
More information about the Ubuntu-openstack-bugs
mailing list