[Bug 2067441] [NEW] [SRU] Loadbalancer is stuck with PENDING_UPDATE state on member update API

[Launchpad Bug Tracker]

You have been subscribed to a public bug by Ubuntu Foundations Team Bug Bot (crichton):

[Impact]

Loadbalancer is stuck with PENDING_UPDATE state on batch member update
API.

[Test Case]

Please refer to [Test steps] section below.

[Regression Potential]

The fix is already in the upstream main, stable/2024.1, stable/2023.2,
stable/2023.1 branches, so it is a clean backport and might be helpful
for deployments using octavia.

I also test this fix, it works well -
https://paste.ubuntu.com/p/wPy7pB3SR6/  and
https://paste.ubuntu.com/p/zpPDScQCtK/

and I also test debdiff for this fix, it works well -
https://paste.ubuntu.com/p/nS6c3QYRGn/


[Others]

Original Bug Description Below
===========

By mistake, I sent wrong request with duplicated ip, port compbination through the Batch Update Members API(ver 2023.1).
https://docs.openstack.org/api-ref/load-balancer/v2/#batch-update-members

For example :
192.0.2.16:80 Member already exists, and request data like follows
{
    "members": [
        {
            "subnet_id": "xxxxxxx",
            "address": "192.0.2.16",
            "protocol_port": 80
        }, {
            "subnet_id": "xxxxxxx",
            "address": "192.0.2.16",
            "protocol_port": 80
        }
    ]
}

After the request, the status of Loadbalancer does not change from
PENDING_UPDATE.

When checking the source code, there is no logic to check for
duplicates.

In the controller logic(member.py), members are classified into
new_members/updated_members/deleted_member, but the updated_members data
is being passed as is with duplicates, so this is suspected to be the
cause of the problem.

## log : 33fe25ab-5477-4787-a8e1-f657376b0ead is duplicated
May 29 04:14:32 ubuntu octavia-worker[123317]: INFO octavia.controller.queue.v2.endpoints [-] Batch updating members: old='[]', new='[]', updated='['825dbebc-da79-4f88-bf48-0e3e63a09d90', '33fe25ab-5477-4787-a8e1-f657376b0ead', '33fe25ab-5477-4787-a8e1-f657376b0ead']'...
May 29 04:14:32 ubuntu octavia-worker[123317]: ERROR oslo_messaging.rpc.server [-] Exception during message handling: taskflow.exceptions.Duplicate: Atoms with duplicate names found: ['octavia-mark-member-active-indb-33fe25ab-5477-4787-a8e1-f657376b0ead']

FYI, There is validation logic for new_members.

[Test steps]

1, set up a openstack env with octavia deployment

2, create a test lb

3, add a member into lb pool

openstack loadbalancer member create --subnet-id private_subnet --address 192.168.21.226 --protocol-port 80 lb1-pool
$ openstack loadbalancer member list lb1-pool |grep ACTIVE
| b36bb21e-8eed-40bc-a1cb-e69da070c0b9 | | 4f1016d73ae245fe8c5c6a637930f3d2 | ACTIVE | 192.168.21.226 | 80 | ONLINE | 1 |

3, run test.py (https://paste.ubuntu.com/p/38vPW5R5S8/) to call batch
member update API to add the same member (eg: 192.168.21.226 above)

4, then we will reproduce the problem, lb will be stuck with
PENDING_UPDATE state.

$ openstack loadbalancer member list lb1-pool |grep 192
| b36bb21e-8eed-40bc-a1cb-e69da070c0b9 | | 4f1016d73ae245fe8c5c6a637930f3d2 | PENDING_UPDATE | 192.168.21.226 | 80 | ONLINE | 40 |

5, This is error log I saw - https://paste.ubuntu.com/p/K5s7knNmWw/

[Some Analyses]

You can see some analysis from the bugs I created earlier -
https://bugs.launchpad.net/octavia/+bug/2070348

** Affects: cloud-archive
     Importance: Undecided
         Status: New

** Affects: cloud-archive/antelope
     Importance: Undecided
         Status: New

** Affects: cloud-archive/bobcat
     Importance: Undecided
         Status: New

** Affects: cloud-archive/caracal
     Importance: Undecided
         Status: New

** Affects: cloud-archive/yoga
     Importance: Undecided
         Status: New

** Affects: cloud-archive/zed
     Importance: Undecided
         Status: New

** Affects: octavia
     Importance: Medium
     Assignee: Gregory Thiemonge (gthiemonge)
         Status: In Progress

** Affects: octavia (Ubuntu)
     Importance: Undecided
         Status: New

** Affects: octavia (Ubuntu Jammy)
     Importance: Undecided
         Status: New

** Affects: octavia (Ubuntu Mantic)
     Importance: Undecided
         Status: New

** Affects: octavia (Ubuntu Noble)
     Importance: Undecided
         Status: New

** Affects: octavia (Ubuntu Oracular)
     Importance: Undecided
         Status: New


** Tags: patch sts
-- 
[SRU] Loadbalancer is stuck with PENDING_UPDATE state on member update API
https://bugs.launchpad.net/bugs/2067441
You received this bug notification because you are a member of Ubuntu Sponsors, which is subscribed to the bug report.