[Bug 1169740] [NEW] rsyslog hangs loading modules

[Launchpad Bug Tracker]

You have been subscribed to a public bug by Adam Stokes (adam-stokes):

[Impact]
We have rsyslog configured to listen for forwarded logs on TCP and UDP, and to forward logs on to a central log host. On a number of occasions we have observed rsyslog in a hung state on our production machines -- not fowarding logs and not writing them to disk locally. It doesn't happen all the time, so I believe there's some sort of race condition involved.

I've attached a stack trace taken when the process was hung. I believe
the issue is the same as the one discussed here: http://www.gossamer-
threads.com/lists/rsyslog/users/5873 There is a patch attached to that
thread.

[Test Case]
#!/bin/sh

while [ 1 ]; do
  service rsyslog stop
  sleep 1
  killall -9 rsyslogd > /dev/null 2>&1
  service rsyslog start
  sleep 1
  nonce=$(date '+%s')
  logger $nonce
  if grep $nonce /var/log/hourly/* > /dev/null; then
    echo found nonce $nonce
  else
    echo FAIL
    exit 1
  fi
done

[Regression Potential]
This is a fix for a deadlock issue it is pretty difficult to determine if fixing a race condition would impact anything else, however, testing from the community hasn't raised any concerns.

[Additional Info]

Ubuntu 12.04.1 LTS x86_64
rsyslog 5.8.6-1ubuntu8

Here is the (obscured) section of our rsyslog config that I believe is
causing the issue.

$ActionQueueType LinkedList
$ActionQueueFileName central_syslog
$ActionQueueMaxDiskSpace 256M
$ActionResumeRetryCount -1
$ActionQueueSaveOnShutdown on
$ActionSendTCPRebindInterval 100000
*.* @@central_syslog.domain:514

$ModLoad imudp
$UDPServerRun 514

$ModLoad imtcp
$InputTCPServerRun 514

** Affects: rsyslog (Ubuntu)
     Importance: Undecided
         Status: Confirmed

** Affects: rsyslog (Ubuntu Precise)
     Importance: Undecided
         Status: New

** Affects: rsyslog (Ubuntu Quantal)
     Importance: Undecided
         Status: New

** Affects: rsyslog (Ubuntu Raring)
     Importance: Undecided
         Status: Confirmed


** Tags: patch precise
-- 
rsyslog hangs loading modules
https://bugs.launchpad.net/bugs/1169740
You received this bug notification because you are a member of Ubuntu Sponsors Team, which is subscribed to the bug report.