[Bug 940030] [NEW] rsyslog stops working after logrotate until restarted

[agent 8131]

Public bug reported:

This could otherwise be titled, rsyslog reload does not create log
files; only restart does.

This is happening on a number of machines I work on.  It's happening on
10.04 and 11.04.  It might be similar to:

https://bugs.launchpad.net/ubuntu/+source/rsyslog/+bug/407862

But in my case after the restart there is no /var/log/syslog being
created, nor auth.log, kern.log, etc.  The files are rotated, rsyslog is
reloaded, and none of the log files are created and nothing is being
logged.  This has been plaguing my systems since moving from syslog-ng,
which I may return to as it seems it was actually production ready.

Without manually restarting those files don't exist so here's what I did
on an 11.04 system:

logrotate --force --verbose /etc/logrotate.conf

gives:

rotating pattern: /var/log/syslog
 forced from command line (7 rotations)
empty log files are not rotated, old logs are removed
considering log /var/log/syslog
  log /var/log/syslog does not exist -- skipping
not running postrotate script, since no logs were rotated

rotating pattern: /var/log/mail.info
/var/log/mail.warn
/var/log/mail.err
/var/log/mail.log
/var/log/daemon.log
/var/log/kern.log
/var/log/auth.log
/var/log/user.log
/var/log/lpr.log
/var/log/cron.log
/var/log/debug
/var/log/messages
 forced from command line (4 rotations)
empty log files are not rotated, old logs are removed
considering log /var/log/mail.info
  log /var/log/mail.info does not exist -- skipping
considering log /var/log/mail.warn
  log /var/log/mail.warn does not exist -- skipping
considering log /var/log/mail.err
  log does not need rotating
considering log /var/log/mail.log
  log does not need rotating
considering log /var/log/daemon.log
  log /var/log/daemon.log does not exist -- skipping
considering log /var/log/kern.log
  log /var/log/kern.log does not exist -- skipping
considering log /var/log/auth.log
  log /var/log/auth.log does not exist -- skipping
considering log /var/log/user.log
  log /var/log/user.log does not exist -- skipping
considering log /var/log/lpr.log
  log /var/log/lpr.log does not exist -- skipping
considering log /var/log/cron.log
  log /var/log/cron.log does not exist -- skipping
considering log /var/log/debug
  log /var/log/debug does not exist -- skipping
considering log /var/log/messages
  log /var/log/messages does not exist -- skipping
not running postrotate script, since no logs were rotated

Then
/sbin/reload rsyslog
logger -i testing

At this point there is no /var/log/syslog

Then:
/sbin/restart rsyslog

And voila there is a /var/log/syslog beginning with:

Feb 23 19:24:48 somehost kernel: imklog 4.6.4, log source = /proc/kmsg started.
Feb 23 19:24:48 somehost rsyslogd: [origin software="rsyslogd" swVersion="4.6.4" x-pid="2299" x-info="http://www.rsyslog.com"] (re)start
Feb 23 19:24:48 somehost rsyslogd: rsyslogd's groupid changed to 114
Feb 23 19:24:48 somehost rsyslogd: rsyslogd's userid changed to 108

Then to recreate:

logrotate --force --verbose /etc/logrotate.conf


rotating pattern: /var/log/syslog
 forced from command line (7 rotations)
empty log files are not rotated, old logs are removed
considering log /var/log/syslog
  log needs rotating
rotating log /var/log/syslog, log->rotateCount is 7
dateext suffix '-20120223'
glob pattern '-[0-9][0-9][0-9][0-9][0-9][0-9][0-9][0-9]'
compressing log with: /bin/gzip
renaming /var/log/syslog to /var/log/syslog-20120223
running postrotate script
removing old log /var/log/syslog-20111219.gz

rotating pattern: /var/log/mail.info
/var/log/mail.warn
/var/log/mail.err
/var/log/mail.log
/var/log/daemon.log
/var/log/kern.log
/var/log/auth.log
/var/log/user.log
/var/log/lpr.log
/var/log/cron.log
/var/log/debug
/var/log/messages
 forced from command line (4 rotations)
empty log files are not rotated, old logs are removed
considering log /var/log/mail.info
  log /var/log/mail.info does not exist -- skipping
considering log /var/log/mail.warn
  log /var/log/mail.warn does not exist -- skipping
considering log /var/log/mail.err
  log does not need rotating
considering log /var/log/mail.log
  log does not need rotating
considering log /var/log/daemon.log
  log /var/log/daemon.log does not exist -- skipping
considering log /var/log/kern.log
  log needs rotating
considering log /var/log/auth.log
  log needs rotating
considering log /var/log/user.log
  log /var/log/user.log does not exist -- skipping
considering log /var/log/lpr.log
  log /var/log/lpr.log does not exist -- skipping
considering log /var/log/cron.log
  log /var/log/cron.log does not exist -- skipping
considering log /var/log/debug
  log /var/log/debug does not exist -- skipping
considering log /var/log/messages
  log /var/log/messages does not exist -- skipping
rotating log /var/log/kern.log, log->rotateCount is 4
dateext suffix '-20120223'
glob pattern '-[0-9][0-9][0-9][0-9][0-9][0-9][0-9][0-9]'
compressing log with: /bin/gzip
rotating log /var/log/auth.log, log->rotateCount is 4
dateext suffix '-20120223'
glob pattern '-[0-9][0-9][0-9][0-9][0-9][0-9][0-9][0-9]'
compressing log with: /bin/gzip
renaming /var/log/kern.log to /var/log/kern.log-20120223
renaming /var/log/auth.log to /var/log/auth.log-20120223
running postrotate script
removing old log /var/log/kern.log-20111218.gz
removing old log /var/log/auth.log-20111218.gz

And, what do you know, there is no more /var/log/syslog, auth.log,
kern.log, etc.

Then /sbin/restart rsyslog and they're there again.  I know from the
other bug permissions were an issue but they seem not to be in this
case:

-rw-r-----  1 syslog   adm            0 2012-02-23 19:29 auth.log
-rw-r-----  1 syslog   adm           79 2012-02-23 19:29 kern.log
-rw-r-----  1 syslog   adm          350 2012-02-23 19:29 syslog

In any case, the solution seems to be updating /etc/logrotate.d/rsyslog

From:
        postrotate
                reload rsyslog >/dev/null 2>&1 || true
        endscript

To:
        postrotate
                /sbin/restart rsyslog >/dev/null 2>&1 || true
        endscript

** Affects: rsyslog (Ubuntu)
     Importance: Undecided
         Status: New

-- 
You received this bug notification because you are a member of Ubuntu
Foundations Bugs, which is subscribed to rsyslog in Ubuntu.
https://bugs.launchpad.net/bugs/940030

Title:
  rsyslog stops working after logrotate until restarted

Status in “rsyslog” package in Ubuntu:
  New

Bug description:
  This could otherwise be titled, rsyslog reload does not create log
  files; only restart does.

  This is happening on a number of machines I work on.  It's happening
  on 10.04 and 11.04.  It might be similar to:

  https://bugs.launchpad.net/ubuntu/+source/rsyslog/+bug/407862

  But in my case after the restart there is no /var/log/syslog being
  created, nor auth.log, kern.log, etc.  The files are rotated, rsyslog
  is reloaded, and none of the log files are created and nothing is
  being logged.  This has been plaguing my systems since moving from
  syslog-ng, which I may return to as it seems it was actually
  production ready.

  Without manually restarting those files don't exist so here's what I
  did on an 11.04 system:

  logrotate --force --verbose /etc/logrotate.conf

  gives:

  rotating pattern: /var/log/syslog
   forced from command line (7 rotations)
  empty log files are not rotated, old logs are removed
  considering log /var/log/syslog
    log /var/log/syslog does not exist -- skipping
  not running postrotate script, since no logs were rotated

  rotating pattern: /var/log/mail.info
  /var/log/mail.warn
  /var/log/mail.err
  /var/log/mail.log
  /var/log/daemon.log
  /var/log/kern.log
  /var/log/auth.log
  /var/log/user.log
  /var/log/lpr.log
  /var/log/cron.log
  /var/log/debug
  /var/log/messages
   forced from command line (4 rotations)
  empty log files are not rotated, old logs are removed
  considering log /var/log/mail.info
    log /var/log/mail.info does not exist -- skipping
  considering log /var/log/mail.warn
    log /var/log/mail.warn does not exist -- skipping
  considering log /var/log/mail.err
    log does not need rotating
  considering log /var/log/mail.log
    log does not need rotating
  considering log /var/log/daemon.log
    log /var/log/daemon.log does not exist -- skipping
  considering log /var/log/kern.log
    log /var/log/kern.log does not exist -- skipping
  considering log /var/log/auth.log
    log /var/log/auth.log does not exist -- skipping
  considering log /var/log/user.log
    log /var/log/user.log does not exist -- skipping
  considering log /var/log/lpr.log
    log /var/log/lpr.log does not exist -- skipping
  considering log /var/log/cron.log
    log /var/log/cron.log does not exist -- skipping
  considering log /var/log/debug
    log /var/log/debug does not exist -- skipping
  considering log /var/log/messages
    log /var/log/messages does not exist -- skipping
  not running postrotate script, since no logs were rotated

  Then
  /sbin/reload rsyslog
  logger -i testing

  At this point there is no /var/log/syslog

  Then:
  /sbin/restart rsyslog

  And voila there is a /var/log/syslog beginning with:

  Feb 23 19:24:48 somehost kernel: imklog 4.6.4, log source = /proc/kmsg started.
  Feb 23 19:24:48 somehost rsyslogd: [origin software="rsyslogd" swVersion="4.6.4" x-pid="2299" x-info="http://www.rsyslog.com"] (re)start
  Feb 23 19:24:48 somehost rsyslogd: rsyslogd's groupid changed to 114
  Feb 23 19:24:48 somehost rsyslogd: rsyslogd's userid changed to 108

  Then to recreate:

  logrotate --force --verbose /etc/logrotate.conf

  
  rotating pattern: /var/log/syslog
   forced from command line (7 rotations)
  empty log files are not rotated, old logs are removed
  considering log /var/log/syslog
    log needs rotating
  rotating log /var/log/syslog, log->rotateCount is 7
  dateext suffix '-20120223'
  glob pattern '-[0-9][0-9][0-9][0-9][0-9][0-9][0-9][0-9]'
  compressing log with: /bin/gzip
  renaming /var/log/syslog to /var/log/syslog-20120223
  running postrotate script
  removing old log /var/log/syslog-20111219.gz

  rotating pattern: /var/log/mail.info
  /var/log/mail.warn
  /var/log/mail.err
  /var/log/mail.log
  /var/log/daemon.log
  /var/log/kern.log
  /var/log/auth.log
  /var/log/user.log
  /var/log/lpr.log
  /var/log/cron.log
  /var/log/debug
  /var/log/messages
   forced from command line (4 rotations)
  empty log files are not rotated, old logs are removed
  considering log /var/log/mail.info
    log /var/log/mail.info does not exist -- skipping
  considering log /var/log/mail.warn
    log /var/log/mail.warn does not exist -- skipping
  considering log /var/log/mail.err
    log does not need rotating
  considering log /var/log/mail.log
    log does not need rotating
  considering log /var/log/daemon.log
    log /var/log/daemon.log does not exist -- skipping
  considering log /var/log/kern.log
    log needs rotating
  considering log /var/log/auth.log
    log needs rotating
  considering log /var/log/user.log
    log /var/log/user.log does not exist -- skipping
  considering log /var/log/lpr.log
    log /var/log/lpr.log does not exist -- skipping
  considering log /var/log/cron.log
    log /var/log/cron.log does not exist -- skipping
  considering log /var/log/debug
    log /var/log/debug does not exist -- skipping
  considering log /var/log/messages
    log /var/log/messages does not exist -- skipping
  rotating log /var/log/kern.log, log->rotateCount is 4
  dateext suffix '-20120223'
  glob pattern '-[0-9][0-9][0-9][0-9][0-9][0-9][0-9][0-9]'
  compressing log with: /bin/gzip
  rotating log /var/log/auth.log, log->rotateCount is 4
  dateext suffix '-20120223'
  glob pattern '-[0-9][0-9][0-9][0-9][0-9][0-9][0-9][0-9]'
  compressing log with: /bin/gzip
  renaming /var/log/kern.log to /var/log/kern.log-20120223
  renaming /var/log/auth.log to /var/log/auth.log-20120223
  running postrotate script
  removing old log /var/log/kern.log-20111218.gz
  removing old log /var/log/auth.log-20111218.gz

  And, what do you know, there is no more /var/log/syslog, auth.log,
  kern.log, etc.

  Then /sbin/restart rsyslog and they're there again.  I know from the
  other bug permissions were an issue but they seem not to be in this
  case:

  -rw-r-----  1 syslog   adm            0 2012-02-23 19:29 auth.log
  -rw-r-----  1 syslog   adm           79 2012-02-23 19:29 kern.log
  -rw-r-----  1 syslog   adm          350 2012-02-23 19:29 syslog

  In any case, the solution seems to be updating
  /etc/logrotate.d/rsyslog

  From:
          postrotate
                  reload rsyslog >/dev/null 2>&1 || true
          endscript

  To:
          postrotate
                  /sbin/restart rsyslog >/dev/null 2>&1 || true
          endscript

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/rsyslog/+bug/940030/+subscriptions