[Bug 1657256] Re: Percona crashes when doing a a 'larger' update

Mon Oct 9 16:41:46 UTC 2017

I found this quite complicated to review, so to break it down I imported
the package into git and then split your debdiff up into multiple git
commits and then applied the relevant quilt patches as commits at the
end so that I could see what was going on. I've pushed this to
https://git.launchpad.net/~racb/ubuntu/+source/percona-xtradb-
cluster-5.6/log/?h=niedbalski&id=niedbalski.c59 and I'll refer to
commits from there in my review as I disect your debdiff.

1. Whitespace cleanups in 6624138 from your previous upload are fine I
suppose, though they did make the debdiff more difficult to review by
creating debdiff noise. But if you're going to clean up like this,
please could you also take control of your trailing whitespace in
debian/changelog in your latest changes in 371f0bc?

2. Quilt noise in 75439ba is fine, though the did make the debdiff more
difficult to review by creating debdiff noise. Please configure your
quilt according to https://wiki.debian.org/UsingQuilt (-p ab --no-
timestamps --no-index) so that quilt patches your generate are always
normalised. This will stop this happening and speed up my reviews.

With those cleaned up, I could then see through the noise through to the
substance of your proposed changes.

3. Commit 18f879d. I thought the purpose of this upload was to stop the
warnings being fatal? Why in addition are we fixing up warnings? I thought this
patch was cherry-picked from upstream? Is this additional change upstream? And
if os_mb is now defined for all architectures, presumably all the calls to
os_mb elsewhere in this patch no longer want to be guarded conditionally on
architecture? I'm not yet sure what I'm asking for here. I think I need to give
the full patch more thorough review (which might impact what's going into the
SRU). If the entire patch can demonstrably only affect ppc64el, then I'd be
happier. I need to check if this is the case.

4. Commit 6842cbe. I understand the disabling of -Werror. Is adding
-fpermissive strictly required or did the build work without? Why is
dropping -Wextra required? And in the diff, why are you dropping -Wall,
-Wextra, -Wunused, -Wwrite-strings, etc? Won't the build still succeed
with those warnings but not being converted to errors with just a drop
of -Werror, or am I missing something?

5. Commit 86ca582. I don't understand the purpose of this change and
debian/changelog doesn't explain it either. Please advise.

6. Commit cc4a8b7. I don't understand the purpose of this change and
debian/changelog doesn't explain it either. Please advise.

7. Commit 371f0bc5. debian/changelog has trailing whitespace. I really
don't care when reviewing others' work, but if you also don't care, then
why clean up trailing whitespace in your quilt patch, creating extra
diff noise and review pain?

-- 
You received this bug notification because you are a member of Ubuntu
Sponsors Team, which is subscribed to the bug report.
https://bugs.launchpad.net/bugs/1657256

Title:
  Percona crashes when doing a a 'larger' update

Status in OpenStack Charm Test Infra:
  Confirmed
Status in percona-xtradb-cluster-5.5 package in Ubuntu:
  Confirmed
Status in percona-xtradb-cluster-5.6 package in Ubuntu:
  In Progress
Status in percona-xtradb-cluster-5.5 source package in Trusty:
  Confirmed
Status in percona-xtradb-cluster-5.6 source package in Xenial:
  Confirmed
Status in percona-xtradb-cluster-5.6 source package in Zesty:
  Confirmed
Status in percona-xtradb-cluster-5.6 source package in Artful:
  In Progress

Bug description:
  [Impact]

   * Percona will segfault when exposed to medium load, almost imediately
   * This is because of a bug in upstream, which manifests only on PPC

  [Test Case]

   * Install and configure percona-xtradb-cluster-server
    * Run sysbench against configured node (only one node is needed, no need for proper cluster):
     # sysbench --test=oltp --oltp-test-mode=complex --max-time=60 --num-threads=110 run
   * mysqld will segfault seconds withing starting the test

  [Regression Potential]

   * This is a cherry-pick from an upstream fix (https://jira.mariadb.org/browse/MDEV-6450)
   * This is not fixed in upstream Percona becasue Percona does not officially support non-intel archs.
   * Because code adds additional memory barriers there was a chance of performance degradation on i386/amd64. However, intensive sysbench syntetic loads proved this is not the case - there are no performance penalties.

  [Other Info]

   *  percona-xtradb-cluster-5.5 is only available for Trusty.
  $ rmadison percona-xtradb-cluster-5.5
   percona-xtradb-cluster-5.5 | 5.5.34-25.9+dfsg-0ubuntu4          | trusty/universe          | source
   percona-xtradb-cluster-5.5 | 5.5.37-25.10+dfsg-0ubuntu0.14.04.1 | trusty-security/universe | source
   percona-xtradb-cluster-5.5 | 5.5.37-25.10+dfsg-0ubuntu0.14.04.2 | trusty-updates/universe  | source

  * See comment #22 for more context about other releases that offers percona-xtradb-cluster-5.6 : 
  https://bugs.launchpad.net/ubuntu/+source/percona-xtradb-cluster-5.5/+bug/1657256/comments/22

   * Upstream commit:
  https://github.com/MariaDB/server/commit/40497577ffd9f85557b15e08ad913f627b2e9530

  [Original Description]

  I'm trying to set up percona-xtradb-cluster-5.5 on PPC machine. While
  the package installs fine, as soon as I run sysbench oltp becnhmark
  against it, Percona dies (even when I start the benchmark with just
  one connection).

  I can also crash mysql manually, by updating the sbtest table (which
  is created by the sysbench utility):

  mysql> update sbtest set pad = 'mario1' limit 1000000;
  ERROR 2013 (HY000): Lost connection to MySQL server during query

  Sometimes I need to repeat this update (with different values for
  'pad' field) few times. This happens regardless of whether I run the
  UPDATE inside the transaction or not.

  This is the assertion found in the log file:

  170117 21:10:55 InnoDB: Assertion failure in thread 70366668321152 in file buf0buf.ic line 1277
  InnoDB: Failing assertion: block->page.buf_fix_count > 0

  This is a single-node percona-xtradb-cluster server, without
  wsrep_provied configured, run inside 14.04 lxc container on 16.04
  host.

  I'm attaching the full log file, mysql configuration file and the core
  dumped.

  The version of the package installed is this 5.5.37-25.10+dfsg-
  0ubuntu0.14.04.2.

To manage notifications about this bug go to:
https://bugs.launchpad.net/charm-test-infra/+bug/1657256/+subscriptions