[Bug 1657256] Re: Percona crashes when doing a a 'larger' update
Jorge Niedbalski
1657256 at bugs.launchpad.net
Tue Oct 17 20:40:11 UTC 2017
Christian, Robbie:
Thanks for taking the time to review the patch that I've submitted. I think that
the latest patch that Christian worked on covers and clean most of the observations made by Robbie
earlier in the bug and I can confirm that it doesn't modifies the original patch with any substantial changes to what I am proposing to fix via this bug.
I've 2 observations in regards to comment #64.
1) I can definitively work with upstream to the fix made in "fix-gcc7-compiler-errors.patch",
I will make a reference to this bug in order to keep all the further work done in upstream in sync with this bug.
2) I have performed testing on top of the PPA[0] and the results seems to work and confirm
the original results of the patch, that means:
a) The sysbench OLTP completes with 50,100 and 200 concurrent threads.
b) No further segfaults are being trapped on the machine for the mysqld process.
A detail on the tests performed:
root at niedbalski-artful:~# dpkg -l | grep percona
ii percona-galera-3 3.19-0ubuntu1 ppc64el Galera replication framework for Percona XtraDB Cluster
ii percona-xtrabackup 2.3.7-0ubuntu2 ppc64el Open source backup tool for InnoDB and XtraDB
ii percona-xtradb-cluster-5.6-dbg 5.6.34-26.19-0ubuntu4~ppa11 ppc64el Debugging package for Percona XtraDB Cluster
ii percona-xtradb-cluster-server 5.6.34-26.19-0ubuntu4~ppa11 all Percona XtraDB Cluster database server
ii percona-xtradb-cluster-server-5.6 5.6.34-26.19-0ubuntu4~ppa11 ppc64el Percona XtraDB Cluster database server binaries
ii percona-xtradb-cluster-source-5.6 5.6.34-26.19-0ubuntu4~ppa11 ppc64el Percona XtraDB Cluster 5.6 source
Version: '5.6.34-79.1-79.1' socket: '/var/run/mysqld/mysqld.sock' port: 3306 Percona XtraDB Cluster (GPL), Release 5.6.34-26.19.4c779b7, wsrep_26.19
~# sysbench --test=oltp --mysql-host=10.75.206.156 --mysql-user=test --mysql-db=dbtest --oltp-test-mode=complex --max-requests=0 --max-time=130 --num-threads=100 run
sysbench 0.4.12: multi-threaded system evaluation benchmark
No DB drivers specified, using mysql
Running the test with following options:
Number of threads: 100
Doing OLTP test.
Running mixed OLTP test
Using Special distribution (12 iterations, 1 pct of values are returned in 75 pct cases)
Using "BEGIN" for starting transactions
Using auto_inc on the id column
Threads started!
Time limit exceeded, exiting...
(last message repeated 99 times)
Done.
OLTP test statistics:
queries performed:
read: 5309948
write: 1794076
other: 719120
total: 7823144
transactions: 339838 (2613.49 per sec.)
deadlocks: 39444 (303.34 per sec.)
read/write requests: 7104024 (54632.84 per sec.)
other operations: 719120 (5530.33 per sec.)
Test execution summary:
total time: 130.0321s
total number of events: 339838
total time taken by event execution: 12996.6388
per-request statistics:
min: 2.65ms
avg: 38.24ms
max: 1348.23ms
approx. 95 percentile: 99.35ms
Threads fairness:
events (avg/stddev): 3398.3800/48.54
execution time (avg/stddev): 129.9664/0.01
---
@cpaelzer, Please let me know if there is any other task that needs to
be performed on my side in order to get this merged/and start the SRU
process for the other series?
Thanks in advance for your comments.
--
You received this bug notification because you are a member of Ubuntu
Sponsors Team, which is subscribed to the bug report.
https://bugs.launchpad.net/bugs/1657256
Title:
Percona crashes when doing a a 'larger' update
Status in OpenStack Charm Test Infra:
Confirmed
Status in percona-xtradb-cluster-5.5 package in Ubuntu:
Confirmed
Status in percona-xtradb-cluster-5.6 package in Ubuntu:
In Progress
Status in percona-xtradb-cluster-5.5 source package in Trusty:
Confirmed
Status in percona-xtradb-cluster-5.6 source package in Xenial:
Confirmed
Status in percona-xtradb-cluster-5.6 source package in Zesty:
Confirmed
Status in percona-xtradb-cluster-5.6 source package in Artful:
In Progress
Bug description:
[Impact]
* Percona will segfault when exposed to medium load, almost imediately
* This is because of a bug in upstream, which manifests only on PPC
[Test Case]
* Install and configure percona-xtradb-cluster-server
* Run sysbench against configured node (only one node is needed, no need for proper cluster):
# sysbench --test=oltp --oltp-test-mode=complex --max-time=60 --num-threads=110 run
* mysqld will segfault seconds withing starting the test
[Regression Potential]
* This is a cherry-pick from an upstream fix (https://jira.mariadb.org/browse/MDEV-6450)
* This is not fixed in upstream Percona becasue Percona does not officially support non-intel archs.
* Because code adds additional memory barriers there was a chance of performance degradation on i386/amd64. However, intensive sysbench syntetic loads proved this is not the case - there are no performance penalties.
[Other Info]
* percona-xtradb-cluster-5.5 is only available for Trusty.
$ rmadison percona-xtradb-cluster-5.5
percona-xtradb-cluster-5.5 | 5.5.34-25.9+dfsg-0ubuntu4 | trusty/universe | source
percona-xtradb-cluster-5.5 | 5.5.37-25.10+dfsg-0ubuntu0.14.04.1 | trusty-security/universe | source
percona-xtradb-cluster-5.5 | 5.5.37-25.10+dfsg-0ubuntu0.14.04.2 | trusty-updates/universe | source
* See comment #22 for more context about other releases that offers percona-xtradb-cluster-5.6 :
https://bugs.launchpad.net/ubuntu/+source/percona-xtradb-cluster-5.5/+bug/1657256/comments/22
* Upstream commit:
https://github.com/MariaDB/server/commit/40497577ffd9f85557b15e08ad913f627b2e9530
[Original Description]
I'm trying to set up percona-xtradb-cluster-5.5 on PPC machine. While
the package installs fine, as soon as I run sysbench oltp becnhmark
against it, Percona dies (even when I start the benchmark with just
one connection).
I can also crash mysql manually, by updating the sbtest table (which
is created by the sysbench utility):
mysql> update sbtest set pad = 'mario1' limit 1000000;
ERROR 2013 (HY000): Lost connection to MySQL server during query
Sometimes I need to repeat this update (with different values for
'pad' field) few times. This happens regardless of whether I run the
UPDATE inside the transaction or not.
This is the assertion found in the log file:
170117 21:10:55 InnoDB: Assertion failure in thread 70366668321152 in file buf0buf.ic line 1277
InnoDB: Failing assertion: block->page.buf_fix_count > 0
This is a single-node percona-xtradb-cluster server, without
wsrep_provied configured, run inside 14.04 lxc container on 16.04
host.
I'm attaching the full log file, mysql configuration file and the core
dumped.
The version of the package installed is this 5.5.37-25.10+dfsg-
0ubuntu0.14.04.2.
To manage notifications about this bug go to:
https://bugs.launchpad.net/charm-test-infra/+bug/1657256/+subscriptions
More information about the Ubuntu-sponsors
mailing list