[Bug 1911900] Re: [SRU] Active scrub blocks upmap balancer

[Łukasz Zemczak]

Hello Ponnuvel, or anyone else affected,

Accepted ceph into groovy-proposed. The package will build now and be
available at
https://launchpad.net/ubuntu/+source/ceph/15.2.8-0ubuntu0.20.10.1 in a
few hours, and then in the -proposed repository.

Please help us by testing this new package.  See
https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how
to enable and use -proposed.  Your feedback will aid us getting this
update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug,
mentioning the version of the package you tested, what testing has been
performed on the package and change the tag from verification-needed-
groovy to verification-done-groovy. If it does not fix the bug for you,
please add a comment stating that, and change the tag to verification-
failed-groovy. In either case, without details of your testing we will
not be able to proceed.

Further information regarding the verification process can be found at
https://wiki.ubuntu.com/QATeam/PerformingSRUVerification .  Thank you in
advance for helping!

N.B. The updated package will be released to -updates after the bug(s)
fixed by this package have been verified and the package has been in
-proposed for a minimum of 7 days.

** Changed in: ceph (Ubuntu Groovy)
       Status: Triaged => Fix Committed

** Tags added: verification-needed verification-needed-groovy

** Changed in: ceph (Ubuntu Focal)
       Status: Triaged => Fix Committed

** Tags added: verification-needed-focal

-- 
You received this bug notification because you are a member of Ubuntu
Sponsors Team, which is subscribed to the bug report.
https://bugs.launchpad.net/bugs/1911900

Title:
  [SRU] Active scrub blocks upmap balancer

Status in Ubuntu Cloud Archive:
  Invalid
Status in Ubuntu Cloud Archive ussuri series:
  Triaged
Status in ceph package in Ubuntu:
  Triaged
Status in ceph source package in Bionic:
  In Progress
Status in ceph source package in Focal:
  Fix Committed
Status in ceph source package in Groovy:
  Fix Committed
Status in ceph source package in Hirsute:
  Triaged

Bug description:
  [Impact]

  When scrubs are in progress, balancer stop due to the bug [0]. And
  shows:

  <timestamp> calc_pg_upmaps abort due to max <= 0

  in the logs.

  Typically when deep-scrub is done in maintenance windows and can take
  few hours. If balancing is paused for the duration, it can affect
  client I/O performance later when balacing starts happening after
  deep-scrub is done.

  This bug was introduced in Octopus. We need to backport upstream bug
  [0] to just Octopus. It's been fixed in upstream master branch [1].

  [Test Case]

  In an Octopus Ceph cluster that has some data (large enough to be able to notice balancing), take down one or more OSDs to introduce "unbalanced" objects.
  Make sure Ceph balancer module is enabled and active (which should be the default case in Octopus).
  Perform some I/O so that data goes to the rest of the OSDs.

  Then start deep-scrubbing and re-add the previously taken down so
  balancing start to happen.

  [Regression potential]

  Low potential. This is a bug fix of a previously correct code.

  If anything goes wrong, the balancer module might not function properly and thus
  leaving the cluster unbalanced and potentially requiring manual balancing.

  [Other Info]

  It's been accepted upstream and backported to Octopus. Ref [0] and
  [1].

  [0] https://tracker.ceph.com/issues/48309
  [1] https://github.com/ceph/ceph/pull/38337

  
  For Hirsute, James is working on a snapshot of Pacific and that should include the fix for this.

To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-archive/+bug/1911900/+subscriptions