[Bug 2092217] Re: [SRU] Missing Group-By Optimization Backport in Yoga and Antelope

Bryan Fraschetti 2092217 at bugs.launchpad.net
Wed Jan 22 14:50:20 UTC 2025 

** Description changed:

- This SRU is related to bug #1918145, which reported slow RBAC queries in
- Neutron. The performance issue was addressed upstream through modifying
- the SQL queries [1] and using a GROUP BY clause in the Neutron lib [2].
+ [ Impact ]
  
- Looking at the Ubuntu Neutron lib package [3], this group by
- optimization clause [2] is missing in stable/yoga and stable/antelope,
- although it present in Zed, Bobcat, and Caracal
+   * There is a missing performance optimization in the Yoga
+ (2.20.0-0ubuntu1) and Antelope (3.4.0-0ubuntu1) releases of neutron-lib
+ caused by a missing database query optimization at [1], which is present
+ upstream. In particular, the commit at [1] minimizes the number of
+ objects returned from the database (grouping by resource id to prevent
+ returning duplicated rows). The commit is a follow up to the commit at
+ [2] in neutron. Both commits are needed to get the performance benefit.
+ Fortunately said neutron commit [2] is present in Yoga
+ (2:20.5.0-0ubuntu2) and Antelope (2:22.0.2-0ubuntu1)
  
- Resources:
- [1] https://review.opendev.org/c/openstack/neutron/+/884877 (Upstream neutron change alongside the library optimization)
+   * To summarize, commit [1] is present in Ubuntu's releases of Zed
+ (3.1.2-0ubuntu1), Bobcat (3.8.0-0ubuntu1), or Caracal (3.11.0-0ubuntu1)
+ but not Yoga (2.20.0-0ubuntu1) or Antelope (3.4.0-0ubuntu1)
  
- [2] https://opendev.org/openstack/neutron-
- lib/commit/829e97024c2b73dd67bfd8a04c65f03be556eec8 (Neutron lib group
- by optimization change that is missing from yoga and antelope)
+   * This proposed SRU patch is a cherry-pick backport of the upstream
+ commit [1]
  
- [3] https://code.launchpad.net/~ubuntu-openstack-
- dev/ubuntu/+source/python-neutron-lib/+git/python-neutron-lib (Ubuntu
- Neutron lib in the UCA)
+ [ Test Plan ]
+ 
+   * Deploy openstack yoga/antelope on jammy and create as many RBAC
+ rules as possible (the original report had 28000). A simple and
+ programmatic way to generate thousands of rules would be to generate
+ thousands of projects, a few admin security groups and networks and
+ share them with all projects
+ 
+   * Measure the time elapsed when executing:
+     openstack network rbac list
+     openstack security group list
+ 
+   * The expected behaviour is that without the patch the performance
+ will degrade with added RBAC groups while there will be no significant
+ performance change with the patch.
+ 
+   [ Where problems could occur ]
+ 
+   * While it is not expected to introduce a regression as the Ubuntu
+ releases surrounding Yoga and Antelope contain the patch, the patch
+ inherently affects performance. There may be edge cases with a reduction
+ in performance. For example, this patch reduces the number of returned
+ values of the query to the set of rows (eliminating duplicates) - if
+ there are few RBAC rules, this may be unnecessary overhead as the query
+ may not have produced many duplicates results to begin with.
+ 
+   * This affects the aggregation of sql queries, if there is an
+ integration that doesn't expect this grouping (for some reason expects
+ duplicates), it may be affected
+ 
+ [ Other Info ]
+ 
+   * There may be questions regarding whether there are accompanying
+ tests to prove that there is no regression. Those tests are present in
+ the associated neutron patch (as per the commit message [3]).
+ 
+   * Note that I have included an additional patch in the Yoga debdiff.
+ This additional patch removes a failing test related to tenant_id. It's
+ not actually related to the neutron patch but I incidentally discovered
+ it when addressing the missing optimization that is the primary concern
+ of this SRU. If one clones the source of the Yoga release and then tries
+ to debuild it (without making any modifications to source), the build
+ fails with a single error associated with a tenant_id test. This test
+ was removed upstream in March 2022 [4], before the Yoga release, shortly
+ after the request context in oslo deprecated the same field in January
+ 2022 [5], which manifested the test failure.
+ 
+ Neutron RBAC performance optimization:
+ [1] https://opendev.org/openstack/neutron-lib/commit/829e97024c2b73dd67bfd8a04c65f03be556eec8
+ [2] https://opendev.org/openstack/neutron/commit/4c654dc553fa8c52e4459c7ac781dca536dba05b
+ [3] https://review.opendev.org/c/openstack/neutron/+/884877
+ 
+ Removal of tenant_id test:
+ [4] https://opendev.org/openstack/neutron-lib/commit/a8abe9d592da5bcf065af40e0ba1cd3599ede1e7
+ [5] https://review.opendev.org/c/openstack/oslo.context/+/815938

-- 
You received this bug notification because you are a member of Ubuntu
OpenStack, which is subscribed to Ubuntu Cloud Archive.
https://bugs.launchpad.net/bugs/2092217

Title:
  [SRU] Missing Group-By Optimization Backport in Yoga and Antelope

Status in Ubuntu Cloud Archive:
  Fix Released
Status in Ubuntu Cloud Archive antelope series:
  New
Status in Ubuntu Cloud Archive bobcat series:
  Fix Released
Status in Ubuntu Cloud Archive caracal series:
  Fix Released
Status in Ubuntu Cloud Archive yoga series:
  New
Status in Ubuntu Cloud Archive zed series:
  Won't Fix
Status in python-neutron-lib package in Ubuntu:
  Fix Released
Status in python-neutron-lib source package in Jammy:
  Confirmed

Bug description:
  [ Impact ]

    * There is a missing performance optimization in the Yoga
  (2.20.0-0ubuntu1) and Antelope (3.4.0-0ubuntu1) releases of neutron-
  lib caused by a missing database query optimization at [1], which is
  present upstream. In particular, the commit at [1] minimizes the
  number of objects returned from the database (grouping by resource id
  to prevent returning duplicated rows). The commit is a follow up to
  the commit at [2] in neutron. Both commits are needed to get the
  performance benefit. Fortunately said neutron commit [2] is present in
  Yoga (2:20.5.0-0ubuntu2) and Antelope (2:22.0.2-0ubuntu1)

    * To summarize, commit [1] is present in Ubuntu's releases of Zed
  (3.1.2-0ubuntu1), Bobcat (3.8.0-0ubuntu1), or Caracal
  (3.11.0-0ubuntu1) but not Yoga (2.20.0-0ubuntu1) or Antelope
  (3.4.0-0ubuntu1)

    * This proposed SRU patch is a cherry-pick backport of the upstream
  commit [1]

  [ Test Plan ]

    * Deploy openstack yoga/antelope on jammy and create as many RBAC
  rules as possible (the original report had 28000). A simple and
  programmatic way to generate thousands of rules would be to generate
  thousands of projects, a few admin security groups and networks and
  share them with all projects

    * Measure the time elapsed when executing:
      openstack network rbac list
      openstack security group list

    * The expected behaviour is that without the patch the performance
  will degrade with added RBAC groups while there will be no significant
  performance change with the patch.

    [ Where problems could occur ]

    * While it is not expected to introduce a regression as the Ubuntu
  releases surrounding Yoga and Antelope contain the patch, the patch
  inherently affects performance. There may be edge cases with a
  reduction in performance. For example, this patch reduces the number
  of returned values of the query to the set of rows (eliminating
  duplicates) - if there are few RBAC rules, this may be unnecessary
  overhead as the query may not have produced many duplicates results to
  begin with.

    * This affects the aggregation of sql queries, if there is an
  integration that doesn't expect this grouping (for some reason expects
  duplicates), it may be affected

  [ Other Info ]

    * There may be questions regarding whether there are accompanying
  tests to prove that there is no regression. Those tests are present in
  the associated neutron patch (as per the commit message [3]).

    * Note that I have included an additional patch in the Yoga debdiff.
  This additional patch removes a failing test related to tenant_id.
  It's not actually related to the neutron patch but I incidentally
  discovered it when addressing the missing optimization that is the
  primary concern of this SRU. If one clones the source of the Yoga
  release and then tries to debuild it, the build fails with a single
  error associated with a tenant_id test. This test was removed upstream
  in March 2022 [4], before the Yoga release, shortly after the request
  context in oslo deprecated the same field in January 2022 [5], which
  manifested the test failure.

  Neutron RBAC performance optimization:
  [1] https://opendev.org/openstack/neutron-lib/commit/829e97024c2b73dd67bfd8a04c65f03be556eec8
  [2] https://opendev.org/openstack/neutron/commit/4c654dc553fa8c52e4459c7ac781dca536dba05b
  [3] https://review.opendev.org/c/openstack/neutron/+/884877

  Removal of tenant_id test:
  [4] https://opendev.org/openstack/neutron-lib/commit/a8abe9d592da5bcf065af40e0ba1cd3599ede1e7
  [5] https://review.opendev.org/c/openstack/oslo.context/+/815938

To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-archive/+bug/2092217/+subscriptions

More information about the Ubuntu-openstack-bugs mailing list