[Bug 2029952] Re: [SRU] backport msgpack max_buffer_size patch to Focal

[Edward Hope-Morley]

wallaby-proposed verified using [Test Plan] and the following output:

root at porygon:/home/ubuntu# apt-cache policy python3-oslo.priv
python3-oslo.privsep:
  Installed: 2.5.0-0ubuntu1~cloud1
  Candidate: 2.5.0-0ubuntu1~cloud1
  Version table:
 *** 2.5.0-0ubuntu1~cloud1 500
        500 http://ubuntu-cloud.archive.canonical.com/ubuntu focal-proposed/wallaby/main amd64 Packages
        100 /var/lib/dpkg/status
     2.1.1-0ubuntu1 500
        500 http://archive.ubuntu.com/ubuntu focal/main amd64 Packages
root at porygon:/home/ubuntu# ip netns exec fip-ec9c32a5-e6e2-4af3-827c-c48452fedc36 ip -br a| wc -l
204
root at porygon:/home/ubuntu# ip netns| grep qrouter| wc -l
202
root at porygon:/home/ubuntu# journalctl --unit neutron-l3-agent --grep ValueError
-- Logs begin at Mon 2023-09-11 10:49:05 UTC, end at Mon 2023-09-11 16:41:16 UTC. --
-- No entries --


** Tags removed: verification-wallaby-needed
** Tags added: verification-wallaby-done

-- 
You received this bug notification because you are a member of Ubuntu
OpenStack, which is subscribed to python-oslo.privsep in Ubuntu.
https://bugs.launchpad.net/bugs/2029952

Title:
  [SRU] backport msgpack max_buffer_size patch to Focal

Status in Ubuntu Cloud Archive:
  Fix Released
Status in Ubuntu Cloud Archive ussuri series:
  Triaged
Status in Ubuntu Cloud Archive victoria series:
  Fix Committed
Status in Ubuntu Cloud Archive wallaby series:
  Fix Committed
Status in Ubuntu Cloud Archive xena series:
  Fix Committed
Status in Ubuntu Cloud Archive yoga series:
  Fix Committed
Status in python-oslo.privsep package in Ubuntu:
  Fix Released
Status in python-oslo.privsep source package in Focal:
  Fix Committed
Status in python-oslo.privsep source package in Jammy:
  Fix Committed

Bug description:
  [Impact]
  Hosts running Openstack Neutron from Ussuri to Yoga release are impacted by
  the fact that their use of oslo.privsep is thwarted by a default buffer size
  on python-msgpack such that commands that return strings > 1MB in size cause
  privsep to crash and agents to stop working. This patch backports a fix that
  increased the buffer size to one that is more appropriate to neutron usage.

  [Test Plan]

    * deploy Openstack (version corresponding to release of SRU) with l3ha
    * need at least one compute host and two neutron-gateway hosts
    * create a large number of routers (200 should do it) each with several networks attached and external gateway set
    * restart neutron-l3-agent
    * wait for restart to complete then do 'journalctl --unit neutron-l3-agent --grep ValueError'
    * if the error does not appear then the patch is working
    * note that it is necessary to create routers with enough networks and ports to trigger the > 1MB size limit for the netlink info message returned by privsep to trigger

  [Where problems could occur]
  No regressions are expected to occur as a result of this patch as it is
  increasing the buffer size by a fairly small amount to allow neutron
  function correctly in loaded environments.

  --------------------------------------------------------------------------------

  The main explanation for this backport can be found in
  https://bugs.launchpad.net/ubuntu/+source/python-
  oslo.privsep/+bug/1896734/comments/37 but I'm opening a new bug for
  the privsep backport since 1896734 was used to backport a neutron fix.

To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-archive/+bug/2029952/+subscriptions