[Bug 1672522] Re: The python-openvswitch package should build the json C extension
Munir Siddiqui
1672522 at bugs.launchpad.net
Mon Oct 12 10:46:59 UTC 2026
Uploading debdiff for jammy
** Patch added: "lp1672522_jammy.debdiff"
https://bugs.launchpad.net/ubuntu/+source/openvswitch/+bug/1672522/+attachment/6008445/+files/lp1672522_jammy.debdiff
--
You received this bug notification because you are a member of Ubuntu
OpenStack, which is subscribed to openvswitch in Ubuntu.
https://bugs.launchpad.net/bugs/1672522
Title:
The python-openvswitch package should build the json C extension
Status in Ubuntu Cloud Archive:
Fix Released
Status in Ubuntu Cloud Archive antelope series:
Fix Released
Status in Ubuntu Cloud Archive bobcat series:
Fix Released
Status in Ubuntu Cloud Archive caracal series:
Fix Released
Status in Ubuntu Cloud Archive dalmatian series:
Fix Released
Status in Ubuntu Cloud Archive epoxy series:
Fix Released
Status in Ubuntu Cloud Archive flamingo series:
Fix Released
Status in Ubuntu Cloud Archive gazpacho series:
Fix Released
Status in Ubuntu Cloud Archive ovn-22.03 series:
In Progress
Status in Ubuntu Cloud Archive yoga series:
In Progress
Status in Ubuntu Cloud Archive zed series:
Fix Released
Status in openvswitch package in Ubuntu:
Fix Released
Status in openvswitch source package in Jammy:
In Progress
Status in openvswitch source package in Noble:
Fix Released
Status in openvswitch source package in Resolute:
Fix Released
Status in openvswitch source package in Stonking:
Fix Released
Bug description:
[ Impact ]
The python3-openvswitch package ships the OVS Python bindings without the
compiled JSON C extension (ovs._json). The extension fails to build during
package compilation and the failure is silently ignored (the build log shows
"WARNING: The C extension could not be compiled, speedups are not enabled"), so
all JSON parsing falls back to the pure-Python parser.
Because every OVSDB transaction must be parsed, JSON parsing is a major
performance bottleneck in the IDL. Under load, particularly in large clouds with
many ports/networks, the pure-Python parser consumes large amounts of CPU in
neutron-server workers and causes API requests to hang or time out while ovsdb
transactions are being processed.
The extension fails to build for two reasons: python/setup.py in OVS 2.17
has no mechanism to receive include paths and linker flags for the C
extension, and the package does not build with -fPIC, which is required to
link the static libopenvswitch into the shared extension module.
The fix backports two upstream OVS commits that enable python/setup.py to
honor the enable_shared/extra_cflags/extra_libs environment variables, and
updates d/rules to build with -fPIC and pass the static linkage flags at
Python install time. python3-openvswitch becomes an arch:any package as it
now ships a compiled module. The upstream commits first appeared in OVS 3.0.0
[0] and 3.1.0 [1] respectively, and the packaging changes mirror what Noble's
openvswitch already does.
[ Test Plan ]
** Setup OpenStack Environment **
1. Deploy a test cloud using stsstack-bundles:
$ cd stsstack-bundles/openstack
$ ./generate-bundle.sh --series jammy --release yoga --name openvswitch-test --run
# wait until all units are active/idle (some will be blocked/waiting)
$ ./tools/vault-unseal-and-authorise.sh
# wait until all units are active/idle
$ ./configure
** Verify the Bug **
2. SSH into the neutron-api unit:
$ juju ssh neutron-api/0
3. Confirm the C extension is not shipped:
$ dpkg -L python3-openvswitch | grep _json
(empty)
4. Confirm the pure-Python parser is active:
$ python3 -c "import ovs._json, ovs.json; print('Active Parser Engine:', ovs.json.PARSER)"
Observed result:
ModuleNotFoundError: No module named 'ovs._json'
5. Confirm neutron-server workers are using the pure-Python parser:
$ for p in $(pgrep -f neutron-server); do sudo grep -q ovs/_json /proc/$p/maps && echo "$p: C" || echo "$p: python"; done
Observed result:
"python" printed for all workers
6. Run the JSON parsing benchmark (see attached benchmark.py, which parses
a synthetic OVSDB payload 10 times):
$ python3 benchmark.py
Observed result:
=== Benchmark Results ===
Host Default Engine: Pure-Python (fallback)
Total Elapsed Time: 13.8047 s
Parsing Throughput: 1.50 MB/s
** Verify the Fix **
7. SSH into the neutron-api unit and install the updated openvswitch packages:
$ juju ssh neutron-api/0
$ sudo apt update && sudo apt upgrade -y openvswitch-common python3-openvswitch
8. Restart neutron-server
$ sudo systemctl restart neutron-server
9. Confirm the C extension is shipped:
$ dpkg -L python3-openvswitch | grep _json
Observed result:
/usr/lib/python3/dist-packages/ovs/_json.cpython-310-x86_64-linux-gnu.so
10. Confirm the C parser is active:
$ python3 -c "import ovs._json, ovs.json; print('Active Parser Engine:', ovs.json.PARSER)"
Observed result:
Active Parser Engine: C
11. Confirm workers load the C extension:
$ for p in $(pgrep -f neutron-server); do sudo grep -q ovs/_json /proc/$p/maps && echo "$p: C" || echo "$p: python"; done
Observed result:
Should show "C" for all workers.
12. Re-run the benchmark:
$ python3 benchmark.py
Observed result:
=== Benchmark Results ===
Host Default Engine: C (ovs._json)
Total Elapsed Time: 0.6149 s
Parsing Throughput: 33.78 MB/s
[ Where problems could occur ]
- The extension is statically linked against libopenvswitch, so no new
runtime library dependencies are introduced; a linkage mistake would
show up as an import failure of ovs._json, caught by step 10.
- The build now uses -fPIC for the whole package. This is the default for
shared objects already and has no functional effect on the produced C
code, but it does rebuild all objects; a toolchain issue would manifest
at build time, not runtime.
- python3-openvswitch changes from Architecture: all to linux-any. Reverse
dependencies were switched from ${source:Version} to ${binary:Version}; any
mistake there would surface as unresolvable dependencies at install time.
- If the extension fails to build in a future rebuild, the upstream fallback
logic silently drops it again, reverting to today's behavior (pure-Python
parser) rather than failing the build.
[ Other Info ]
The two setup.py commits are upstream: 671f93f ("python: Allow building json C
extension with static OVS library.") in OVS 3.0.0 [0], and 791c472 ("python: Add
ability to pass extra libs and cflags for C extension.") in OVS 3.1.0 [1].
The d/rules and d/control changes mirror three Debian packaging commits:
"d/rules: Build the Python JSON C extension" [2], "d/control: Make
python3-openvswitch a binary package, drop python3-six dependency" [3], and
"d/rules: Add comment about the explicit -fPIC for this package" [4]. These
first appeared in Debian's openvswitch 3.0.1-1.
Ubuntu releases Noble onward and the OVN/OVS versions shipped in corresponding
UCA pockets already ship the compiled extension, so only Jammy needs this fix.
Upstream OVS 2.17 is EOL, so the setup.py changes cannot be backported there.
A test package was built and is available in the following PPA:
https://launchpad.net/~munirsid/+archive/ubuntu/sf427278-test
[0] - https://github.com/openvswitch/ovs/commit/671f93fe42d3c15aa0bf883bef0f05716bc2b18b
[1] - https://github.com/openvswitch/ovs/commit/791c47273983b830dbc840ae7bf18a23bc9e72e4
[2] - https://salsa.debian.org/openstack-team/third-party/openvswitch/-/commit/45734ca10e2ac97fe2646588596933adb00f81ba
[3] - https://salsa.debian.org/openstack-team/third-party/openvswitch/-/commit/22f00e3fcfece1753d4cde17c9ae0b9394a4a966
[4] - https://salsa.debian.org/openstack-team/third-party/openvswitch/-/commit/b120fc6a2caad12a7225c7600172f90102c0e414
Original Description: Find the original description of the bug below:
The C extension wrapping OVS's own json parser is 60x faster than the
pure-python version currently being used. Profiling shows JSON parsing
to be the biggest bottleneck using python-openvswitch IDL library.
setuptools can build the extension as long as the python-dev packages
are installed and openvswitch-dev is installed. It should be available
in OVS 2.6+
To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-archive/+bug/1672522/+subscriptions
More information about the Ubuntu-openstack-bugs
mailing list