[Bug 1840465] Re: [SRU] Fails to list security groups if one or more exists without rules

Tue Nov 12 23:12:16 UTC 2019

While performing validation, I was unable to reproduce the issue in
bionic-rocky deployment. Inspecting the code, the fix was not present. I
added logging, and confirmed that neutron is now including the key
"security_group_rules" in the response, therefore the error is no longer
triggered. I added logging to confirm that it was indeed the case, and
got:

[Tue Nov 12 22:32:51.766942 2019] [wsgi:error] [pid 26153:tid
139930429884160] [remote 10.5.0.43:40358] sg: {'id': '40ca8baa-
a3a5-412d-b955-7f9fb3e86e26', 'name': 'test3', 'tenant_id':
'a3ef2c1a2a8e4b628f592e0f87ef3c11', 'description': '',
'security_group_rules': [], 'tags': [], 'created_at':
'2019-11-12T22:11:56Z', 'updated_at': '2019-11-12T22:12:03Z',
'revision_number': 3, 'project_id': 'a3ef2c1a2a8e4b628f592e0f87ef3c11'}

I also re-tested this with a limited user account, same results.

I proceeded to upgrade the package, confirmed the code has been changed
and includes that fix. Ran the test case and saw that it does not cause
any problems. Added additional logging to confirm the new condition (the
fix) is not invoked as it is not necessary and indeed it was not being
invoked.

I briefly checked neutron, python-neutronclient and horizon code, but I
haven't been able to track down the change that caused the key
'security_group_rules' to no longer be missing.

I proceeded to reproduce the problem in xenial-queens to re-confirm the
test case, and I couldn't.

Therefore my conclusion is that the test case needs to be improved, as
it seems to be missing details that are required to reproduce the
problem.

-- 
You received this bug notification because you are a member of Ubuntu
OpenStack, which is subscribed to horizon in Ubuntu.
https://bugs.launchpad.net/bugs/1840465

Title:
  [SRU] Fails to list security groups if one or more exists without
  rules

Status in Ubuntu Cloud Archive:
  Fix Released
Status in Ubuntu Cloud Archive queens series:
  Triaged
Status in Ubuntu Cloud Archive rocky series:
  Fix Committed
Status in Ubuntu Cloud Archive stein series:
  Triaged
Status in Ubuntu Cloud Archive train series:
  Fix Released
Status in OpenStack Dashboard (Horizon):
  Fix Released
Status in horizon package in Ubuntu:
  Fix Released
Status in horizon source package in Bionic:
  In Progress
Status in horizon source package in Disco:
  In Progress
Status in horizon source package in Eoan:
  Fix Released

Bug description:
  Horizon 14.0.2 (rocky)
  If a security group without any rules exists the listing of security groups fails with a KeyError.

  Traceback (most recent call last):
    File "/usr/share/openstack-dashboard/openstack_dashboard/api/rest/utils.py", line 127, in _wrapped
      data = function(self, request, *args, **kw)
    File "/usr/share/openstack-dashboard/openstack_dashboard/api/rest/network.py", line 44, in get
      security_groups = api.neutron.security_group_list(request)
    File "/usr/lib/python2.7/site-packages/horizon/utils/memoized.py", line 95, in wrapped
      value = cache[key] = func(*args, **kwargs)
    File "/usr/share/openstack-dashboard/openstack_dashboard/api/neutron.py", line 1641, in security_group_list
      return SecurityGroupManager(request).list(**params)
    File "/usr/share/openstack-dashboard/openstack_dashboard/api/neutron.py", line 372, in list
      return self._list(**params)
    File "/usr/share/openstack-dashboard/openstack_dashboard/api/neutron.py", line 359, in _list
      return [SecurityGroup(sg) for sg in secgroups.get('security_groups')]
    File "/usr/share/openstack-dashboard/openstack_dashboard/api/neutron.py", line 240, in __init__
      for rule in sg['security_group_rules']]
  KeyError: 'security_group_rules'

  =======================================================================

  [Impact]

  By default, new security groups created through horizon or CLI include
  2 default security rules. Upon managing those rules and removing them
  (to perhaps add others or limit traffic completely), the security
  group page errors out and prevents listing of *all* security groups if
  the empty security group is within the list to be displayed.
  Therefore, not only is the empty security group affected, but all
  others as well, as they cannot be listed. The root cause of the bug is
  that the payload does not include the expected key
  "security_group_rules" for that security group when there are no
  rules.

  A fix has been implemented for Train (from master), Stein, Rocky and
  Queens releases and should be backported so the issue is addressed on
  previous those releases. The fix prevents the crash by ensuring the
  key "security_group_rules" is present with an empty list in case it
  was not included in the payload.

  [Test Case]

  1. Reproducing the issue

  1a. Go to the Security Group section at Project > Network > Security Groups
  1b. Create a security group
  1c. Click the Manage Rules button for that security group you just created
  1d. Delete the two default rules
  1e. Go back to the Security Group section at Project > Network > Security Groups
  1f. Security groups are no longer being listed and there will be an error popup: "Error: Unable to retrieve security groups.".

  2. Install the package with the fixed code

  3. Confirm bug has been fixed

  3a. Repeat step 1a, you will notice that now security groups can be
  listed, including the empty one you had previously created

  3b. Repeat steps 1b through 1e, the newly created and emptied security
  group can also be listed along the others.

  [Regression Potential]

  Given the following indicators:

  a. Upstream CI passed, for all releases the fix is being backported.
  b. The small scope of the problem: a key error prevents the page from being rendered. The fix is to make sure the key is always present, since it is always expected.
  c. The fix is very simple, and during tests we can see that the other pages handle an empty list properly as the value for the "security_group_rules" key.

  We can safely state that the regression potential is negligible.

To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-archive/+bug/1840465/+subscriptions