[Bug 940030] Re: rsyslog stops working after logrotate until restarted

Tue Sep 3 17:19:47 UTC 2013

One of the forum mods (and likely of other official roles) over at the
rsyslog.com knowledge-base suggests in this relatively recent thread:

http://kb.monitorware.com/local-file-logging-stopped-t11391.html

.. that this condition is likely reflecting a per-distribution,
implementation-specific problem related to the configuration options

$PrivDropToUser [user]
$PrivDropToGroup [group]

For rsyslog 4.7.x, and 5.3.x to 5.8.x, their docs suggest there is a
built-in workaround that is likely effective in many cases,
'$omfileForceChown which can be added to the rsyslog.conf:

$omfileForceChown on

http://www.rsyslog.com/doc/rsconf1_omfileforcechown.html

Note this workaround is not available in subsequent versions of rsyslog
(eg upstream ppa's such as the one they host at rsyslog.com).

To me, it seems that this process could be simplified by creating the
replacement file before removing/transitioning away from/stashing the
old log file and then just guessing at the permissions.  This should let
correct/desired and specific ownership and permission masks be
copied/applied with the least complication instead of relying on suid,
sguid, configuration global generic/universal masks, or other sometimes
tricky or perilous techniques.  A possible caveat is that the existing
log file that rsyslog is writing to, must also allow reading in order to
get necessary security meta-information.  Ideally, the configuration
option would only be a fall-through option if the log didn't exist in
the first place (essentially a stricter take on an independent service
facility provider umask).

By utilizing either "cp --attributes-only" from a recent core-utils
update (16/17) before remove/stash, this particular issue might be
permanently worked around.. or by using 'stat' with formatting options
to read ownership and permissions before remove/stash, such as:

cp --attributes-only "$log_to_stash" "$log_to_activate"
rotate_cmd "$log_to_stash" "$log_to_activate"

or

elevate_up
touch "$log_to_activate"
chmod `stat --printf="%a" $log_to_stash` "$log_to_activate"
chown `stat --printf="%U:%G" $log_to_stash` "$log_to_activate"
rotate_cmd "$log_to_stash" "$log_to_activate"
drop_back_cmd

Note that "cp --attributes-only" is a fairly recent addition to core-utils, and I am sure among many other places, discussed briefly here:
https://bugzilla.redhat.com/show_bug.cgi?id=811746

'cp --attributes-only' does not seem to rely on the same underlying
behavior of chown and currently seems able to set target meta ownership
information even if executor is not of elevated privilege (as it is
apparently and relatively new, I am not sure what the intended behavior
is in this case).  Likely it is intentionally permitted as no underlying
data is coming with the file.

** Bug watch added: Red Hat Bugzilla #811746
   https://bugzilla.redhat.com/show_bug.cgi?id=811746

-- 
You received this bug notification because you are a member of Ubuntu
Foundations Bugs, which is subscribed to rsyslog in Ubuntu.
https://bugs.launchpad.net/bugs/940030

Title:
  rsyslog stops working after logrotate until restarted

Status in “rsyslog” package in Ubuntu:
  Confirmed

Bug description:
  This could otherwise be titled, rsyslog reload does not create log
  files; only restart does.

  This is happening on a number of machines I work on.  It's happening
  on 10.04 and 11.04.  It might be similar to:

  https://bugs.launchpad.net/ubuntu/+source/rsyslog/+bug/407862

  But in my case after the restart there is no /var/log/syslog being
  created, nor auth.log, kern.log, etc.  The files are rotated, rsyslog
  is reloaded, and none of the log files are created and nothing is
  being logged.  This has been plaguing my systems since moving from
  syslog-ng, which I may return to as it seems it was actually
  production ready.

  Without manually restarting those files don't exist so here's what I
  did on an 11.04 system:

  logrotate --force --verbose /etc/logrotate.conf

  gives:

  rotating pattern: /var/log/syslog
   forced from command line (7 rotations)
  empty log files are not rotated, old logs are removed
  considering log /var/log/syslog
    log /var/log/syslog does not exist -- skipping
  not running postrotate script, since no logs were rotated

  rotating pattern: /var/log/mail.info
  /var/log/mail.warn
  /var/log/mail.err
  /var/log/mail.log
  /var/log/daemon.log
  /var/log/kern.log
  /var/log/auth.log
  /var/log/user.log
  /var/log/lpr.log
  /var/log/cron.log
  /var/log/debug
  /var/log/messages
   forced from command line (4 rotations)
  empty log files are not rotated, old logs are removed
  considering log /var/log/mail.info
    log /var/log/mail.info does not exist -- skipping
  considering log /var/log/mail.warn
    log /var/log/mail.warn does not exist -- skipping
  considering log /var/log/mail.err
    log does not need rotating
  considering log /var/log/mail.log
    log does not need rotating
  considering log /var/log/daemon.log
    log /var/log/daemon.log does not exist -- skipping
  considering log /var/log/kern.log
    log /var/log/kern.log does not exist -- skipping
  considering log /var/log/auth.log
    log /var/log/auth.log does not exist -- skipping
  considering log /var/log/user.log
    log /var/log/user.log does not exist -- skipping
  considering log /var/log/lpr.log
    log /var/log/lpr.log does not exist -- skipping
  considering log /var/log/cron.log
    log /var/log/cron.log does not exist -- skipping
  considering log /var/log/debug
    log /var/log/debug does not exist -- skipping
  considering log /var/log/messages
    log /var/log/messages does not exist -- skipping
  not running postrotate script, since no logs were rotated

  Then
  /sbin/reload rsyslog
  logger -i testing

  At this point there is no /var/log/syslog

  Then:
  /sbin/restart rsyslog

  And voila there is a /var/log/syslog beginning with:

  Feb 23 19:24:48 somehost kernel: imklog 4.6.4, log source = /proc/kmsg started.
  Feb 23 19:24:48 somehost rsyslogd: [origin software="rsyslogd" swVersion="4.6.4" x-pid="2299" x-info="http://www.rsyslog.com"] (re)start
  Feb 23 19:24:48 somehost rsyslogd: rsyslogd's groupid changed to 114
  Feb 23 19:24:48 somehost rsyslogd: rsyslogd's userid changed to 108

  Then to recreate:

  logrotate --force --verbose /etc/logrotate.conf

  rotating pattern: /var/log/syslog
   forced from command line (7 rotations)
  empty log files are not rotated, old logs are removed
  considering log /var/log/syslog
    log needs rotating
  rotating log /var/log/syslog, log->rotateCount is 7
  dateext suffix '-20120223'
  glob pattern '-[0-9][0-9][0-9][0-9][0-9][0-9][0-9][0-9]'
  compressing log with: /bin/gzip
  renaming /var/log/syslog to /var/log/syslog-20120223
  running postrotate script
  removing old log /var/log/syslog-20111219.gz

  rotating pattern: /var/log/mail.info
  /var/log/mail.warn
  /var/log/mail.err
  /var/log/mail.log
  /var/log/daemon.log
  /var/log/kern.log
  /var/log/auth.log
  /var/log/user.log
  /var/log/lpr.log
  /var/log/cron.log
  /var/log/debug
  /var/log/messages
   forced from command line (4 rotations)
  empty log files are not rotated, old logs are removed
  considering log /var/log/mail.info
    log /var/log/mail.info does not exist -- skipping
  considering log /var/log/mail.warn
    log /var/log/mail.warn does not exist -- skipping
  considering log /var/log/mail.err
    log does not need rotating
  considering log /var/log/mail.log
    log does not need rotating
  considering log /var/log/daemon.log
    log /var/log/daemon.log does not exist -- skipping
  considering log /var/log/kern.log
    log needs rotating
  considering log /var/log/auth.log
    log needs rotating
  considering log /var/log/user.log
    log /var/log/user.log does not exist -- skipping
  considering log /var/log/lpr.log
    log /var/log/lpr.log does not exist -- skipping
  considering log /var/log/cron.log
    log /var/log/cron.log does not exist -- skipping
  considering log /var/log/debug
    log /var/log/debug does not exist -- skipping
  considering log /var/log/messages
    log /var/log/messages does not exist -- skipping
  rotating log /var/log/kern.log, log->rotateCount is 4
  dateext suffix '-20120223'
  glob pattern '-[0-9][0-9][0-9][0-9][0-9][0-9][0-9][0-9]'
  compressing log with: /bin/gzip
  rotating log /var/log/auth.log, log->rotateCount is 4
  dateext suffix '-20120223'
  glob pattern '-[0-9][0-9][0-9][0-9][0-9][0-9][0-9][0-9]'
  compressing log with: /bin/gzip
  renaming /var/log/kern.log to /var/log/kern.log-20120223
  renaming /var/log/auth.log to /var/log/auth.log-20120223
  running postrotate script
  removing old log /var/log/kern.log-20111218.gz
  removing old log /var/log/auth.log-20111218.gz

  And, what do you know, there is no more /var/log/syslog, auth.log,
  kern.log, etc.

  Then /sbin/restart rsyslog and they're there again.  I know from the
  other bug permissions were an issue but they seem not to be in this
  case:

  -rw-r-----  1 syslog   adm            0 2012-02-23 19:29 auth.log
  -rw-r-----  1 syslog   adm           79 2012-02-23 19:29 kern.log
  -rw-r-----  1 syslog   adm          350 2012-02-23 19:29 syslog

  In any case, the solution seems to be updating
  /etc/logrotate.d/rsyslog

  From:
          postrotate
                  reload rsyslog >/dev/null 2>&1 || true
          endscript

  To:
          postrotate
                  /sbin/restart rsyslog >/dev/null 2>&1 || true
          endscript

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/rsyslog/+bug/940030/+subscriptions