[Bug 1905423] Re: Rabbit reports "file descriptor limit alarm set", does not accept connections

[Peter Sabaini]

I unfortunately do not have a way to reproduce this. This indeed
happened in a large-ish production env with lots of clients. It's the
first time I saw this.

One thing I'm wondering, if rabbit were started with

$ sudo rabbitmqctl start_app

the systemd unit file probably would not go into effect, and the default
limits would apply, right?

It's possible that by way of troubleshooting this rabbit instance had
been started up via rabbitmqctl (I don't know for sure though).

-- 
You received this bug notification because you are a member of Ubuntu
OpenStack, which is subscribed to rabbitmq-server in Ubuntu.
https://bugs.launchpad.net/bugs/1905423

Title:
  Rabbit reports "file descriptor limit alarm set", does not accept
  connections

Status in rabbitmq-server package in Ubuntu:
  New

Bug description:
  We had an incident where rabbitmq-server would stop accepting
  connections, with the following message in the log:

  =WARNING REPORT==== 24-Nov-2020::09:46:02 ===
  file descriptor limit alarm set.

  ********************************************************************
  *** New connections will not be accepted until this alarm clears ***
  ********************************************************************

  
  However, when checking the fd limits these are set quite high:

  root at juju-efcc41-12-lxd-0:~# cat /etc/default/rabbitmq-server
  # Generated by juju
  # bump ulimit so rabbit can support lots of connections
  ulimit -n 65536

  root at juju-efcc41-12-lxd-0:~# grep Limit /lib/systemd/system/rabbitmq-server.service
  LimitNOFILE=65536

  On the other hand, checking fd usage via lsof, `lsof -u rabbitmq`
  would only report abt. 900 open fds

  After bouncing rabbitmq it resumed accepting connects

  Versions:

  rabbitmq-server                  3.5.7-1ubuntu0.16.04.2 
  Ubuntu 16.04.5 LTS

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/rabbitmq-server/+bug/1905423/+subscriptions