[Bug 1783203] Re: Upgrade to RabbitMQ 3.6.10 causes beam lockup in clustered deployment
Dr. Jens Harbott
j.harbott at x-ion.de
Mon Feb 25 11:58:36 UTC 2019
Forgot to mention: The log in /var/log/rabbitmq/rabbit at hostname.log just
stops when the server hangs, no output until after it is being
restarted.
--
You received this bug notification because you are a member of Ubuntu
OpenStack, which is subscribed to rabbitmq-server in Ubuntu.
https://bugs.launchpad.net/bugs/1783203
Title:
Upgrade to RabbitMQ 3.6.10 causes beam lockup in clustered deployment
Status in OpenStack rabbitmq-server charm:
New
Status in rabbitmq-server package in Ubuntu:
Confirmed
Bug description:
While performing an openstack release upgrade from Pike to Queens
following the charmers guide, we have upgraded Ceph-* and Mysql.
After setting source=cloud:xenial-queens on the RabbitMQ-Server charm
and the cluster re-stabilizes, rabbitmq beam processes lock up on one
cluster node causing complete denial of service on the openstack vhost
across all 3 members of the cluster. Killing the beam process on that
node causes another node to lock up within a short timeframe.
We have reproduced this twice in the same environment by re-deploying
a fresh pike rabbitmq cluster and upgrading to queens. The issue is
not reproducable with generic workloads such as creating/deleting nova
instances and creating/attaching/detaching cinder volumes, however,
when running a full heat stack, we can reproduce this issue.
This is happening on two of the three clouds on this site when RMQ is
upgraded to Queens. The third cloud was able to upgrade to Queens
without issue but was upgraded on 18.02 charms. Heat templates
forthcoming.
To manage notifications about this bug go to:
https://bugs.launchpad.net/charm-rabbitmq-server/+bug/1783203/+subscriptions
More information about the Ubuntu-openstack-bugs
mailing list