[Bug 1642538] Re: swift-ring-builder rebalance - endless loop

Wed Dec 7 01:10:31 UTC 2016

Reviewed:  https://review.openstack.org/399723
Committed: https://git.openstack.org/cgit/openstack/swift/commit/?id=17f6a6989d09d3d7522efcedf747d773d11d2535
Submitter: Jenkins
Branch:    stable/mitaka

commit 17f6a6989d09d3d7522efcedf747d773d11d2535
Author: Tim Burke <tim.burke at gmail.com>
Date:   Thu Nov 17 13:02:06 2016 -0800

    Avoid infinite loop while placing parts

    Previously, we could over-assign how many parts should be in a tier.
    This would cause the local `parts` variable to go negative, which meant
    that our `while parts` loop would never terminate.

    Change-Id: Id7e7889742ca37cf1a9c0d55fba78d967e90e8d0
    Closes-Bug: 1642538
    (cherry picked from commit 2e7a7347fc58676fbaabce3d87a15866796d32e4)

** Changed in: cloud-archive/mitaka
       Status: Triaged => Fix Committed

-- 
You received this bug notification because you are a member of Ubuntu
OpenStack, which is subscribed to swift in Ubuntu.
https://bugs.launchpad.net/bugs/1642538

Title:
  swift-ring-builder rebalance - endless loop

Status in Ubuntu Cloud Archive:
  Triaged
Status in Ubuntu Cloud Archive mitaka series:
  Fix Committed
Status in Ubuntu Cloud Archive newton series:
  Triaged
Status in Ubuntu Cloud Archive ocata series:
  Triaged
Status in OpenStack Object Storage (swift):
  Fix Released
Status in swift package in Ubuntu:
  Triaged
Status in swift source package in Xenial:
  Triaged
Status in swift source package in Yakkety:
  Triaged
Status in swift source package in Zesty:
  Triaged

Bug description:
  There is an issue with swift-ring-builder and rebalancing under
  certain conditions beginning in Mitaka.

  Steps to reproduce:
  Create a ring with part power of 12. 3 servers with 11 devices each, BUT 2 servers in zone 1 and one server in zone 2. After adding all devices as described above, swift-ring-builder <ring>.builder rebalance does not return and seems to run into an endless loop.

  The weirdness starts with the 11th device, 10 devices still work. It
  does not matter whether you add each 10 devices from the 3 servers and
  rebalance, and than add the 11th device from each server afterwards or
  do it all at once.

  This behaviour does not exists in Liberty and starts with Mitaka.

  For connivence I am adding the statements to reproduce:

  swift-ring-builder account.builder create 12 3 1

  swift-ring-builder account.builder add --region 1 --zone 1 --ip 10.46.15.60 --port 6002 --replication-ip 10.46.15.60 --replication-port 6002 --device swift-01 --weight 100.00
  swift-ring-builder account.builder add --region 1 --zone 1 --ip 10.46.15.60 --port 6002 --replication-ip 10.46.15.60 --replication-port 6002 --device swift-02 --weight 100.00
  swift-ring-builder account.builder add --region 1 --zone 1 --ip 10.46.15.60 --port 6002 --replication-ip 10.46.15.60 --replication-port 6002 --device swift-03 --weight 100.00
  swift-ring-builder account.builder add --region 1 --zone 1 --ip 10.46.15.60 --port 6002 --replication-ip 10.46.15.60 --replication-port 6002 --device swift-04 --weight 100.00
  swift-ring-builder account.builder add --region 1 --zone 1 --ip 10.46.15.60 --port 6002 --replication-ip 10.46.15.60 --replication-port 6002 --device swift-05 --weight 100.00
  swift-ring-builder account.builder add --region 1 --zone 1 --ip 10.46.15.60 --port 6002 --replication-ip 10.46.15.60 --replication-port 6002 --device swift-06 --weight 100.00
  swift-ring-builder account.builder add --region 1 --zone 1 --ip 10.46.15.60 --port 6002 --replication-ip 10.46.15.60 --replication-port 6002 --device swift-07 --weight 100.00
  swift-ring-builder account.builder add --region 1 --zone 1 --ip 10.46.15.60 --port 6002 --replication-ip 10.46.15.60 --replication-port 6002 --device swift-08 --weight 100.00
  swift-ring-builder account.builder add --region 1 --zone 1 --ip 10.46.15.60 --port 6002 --replication-ip 10.46.15.60 --replication-port 6002 --device swift-09 --weight 100.00
  swift-ring-builder account.builder add --region 1 --zone 1 --ip 10.46.15.60 --port 6002 --replication-ip 10.46.15.60 --replication-port 6002 --device swift-10 --weight 100.00
  swift-ring-builder account.builder add --region 1 --zone 1 --ip 10.46.15.60 --port 6002 --replication-ip 10.46.15.60 --replication-port 6002 --device swift-11 --weight 100.00

  swift-ring-builder account.builder add --region 1 --zone 1 --ip 10.46.15.61 --port 6002 --replication-ip 10.46.15.61 --replication-port 6002 --device swift-01 --weight 100.00
  swift-ring-builder account.builder add --region 1 --zone 1 --ip 10.46.15.61 --port 6002 --replication-ip 10.46.15.61 --replication-port 6002 --device swift-02 --weight 100.00
  swift-ring-builder account.builder add --region 1 --zone 1 --ip 10.46.15.61 --port 6002 --replication-ip 10.46.15.61 --replication-port 6002 --device swift-03 --weight 100.00
  swift-ring-builder account.builder add --region 1 --zone 1 --ip 10.46.15.61 --port 6002 --replication-ip 10.46.15.61 --replication-port 6002 --device swift-04 --weight 100.00
  swift-ring-builder account.builder add --region 1 --zone 1 --ip 10.46.15.61 --port 6002 --replication-ip 10.46.15.61 --replication-port 6002 --device swift-05 --weight 100.00
  swift-ring-builder account.builder add --region 1 --zone 1 --ip 10.46.15.61 --port 6002 --replication-ip 10.46.15.61 --replication-port 6002 --device swift-06 --weight 100.00
  swift-ring-builder account.builder add --region 1 --zone 1 --ip 10.46.15.61 --port 6002 --replication-ip 10.46.15.61 --replication-port 6002 --device swift-07 --weight 100.00
  swift-ring-builder account.builder add --region 1 --zone 1 --ip 10.46.15.61 --port 6002 --replication-ip 10.46.15.61 --replication-port 6002 --device swift-08 --weight 100.00
  swift-ring-builder account.builder add --region 1 --zone 1 --ip 10.46.15.61 --port 6002 --replication-ip 10.46.15.61 --replication-port 6002 --device swift-09 --weight 100.00
  swift-ring-builder account.builder add --region 1 --zone 1 --ip 10.46.15.61 --port 6002 --replication-ip 10.46.15.61 --replication-port 6002 --device swift-10 --weight 100.00
  swift-ring-builder account.builder add --region 1 --zone 1 --ip 10.46.15.61 --port 6002 --replication-ip 10.46.15.61 --replication-port 6002 --device swift-11 --weight 100.00

  swift-ring-builder account.builder add --region 1 --zone 2 --ip 10.46.15.62 --port 6002 --replication-ip 10.46.15.62 --replication-port 6002 --device swift-01 --weight 100.00
  swift-ring-builder account.builder add --region 1 --zone 2 --ip 10.46.15.62 --port 6002 --replication-ip 10.46.15.62 --replication-port 6002 --device swift-02 --weight 100.00
  swift-ring-builder account.builder add --region 1 --zone 2 --ip 10.46.15.62 --port 6002 --replication-ip 10.46.15.62 --replication-port 6002 --device swift-03 --weight 100.00
  swift-ring-builder account.builder add --region 1 --zone 2 --ip 10.46.15.62 --port 6002 --replication-ip 10.46.15.62 --replication-port 6002 --device swift-04 --weight 100.00
  swift-ring-builder account.builder add --region 1 --zone 2 --ip 10.46.15.62 --port 6002 --replication-ip 10.46.15.62 --replication-port 6002 --device swift-05 --weight 100.00
  swift-ring-builder account.builder add --region 1 --zone 2 --ip 10.46.15.62 --port 6002 --replication-ip 10.46.15.62 --replication-port 6002 --device swift-06 --weight 100.00
  swift-ring-builder account.builder add --region 1 --zone 2 --ip 10.46.15.62 --port 6002 --replication-ip 10.46.15.62 --replication-port 6002 --device swift-07 --weight 100.00
  swift-ring-builder account.builder add --region 1 --zone 2 --ip 10.46.15.62 --port 6002 --replication-ip 10.46.15.62 --replication-port 6002 --device swift-08 --weight 100.00
  swift-ring-builder account.builder add --region 1 --zone 2 --ip 10.46.15.62 --port 6002 --replication-ip 10.46.15.62 --replication-port 6002 --device swift-09 --weight 100.00
  swift-ring-builder account.builder add --region 1 --zone 2 --ip 10.46.15.62 --port 6002 --replication-ip 10.46.15.62 --replication-port 6002 --device swift-10 --weight 100.00
  swift-ring-builder account.builder add --region 1 --zone 2 --ip 10.46.15.62 --port 6002 --replication-ip 10.46.15.62 --replication-port 6002 --device swift-11 --weight 100.00

  swift-ring-builder account.builder rebalance

To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-archive/+bug/1642538/+subscriptions