[Bug 1618188] Re: systemd journal should be persistent by default: /var/log/journal should be created

[Bryan Quigley]

@markstos
Sorry, yea, I meant our defaults, not the journal config options itself. SystemMaxUse= is unset in the config in bionic (although it's all commented out, but I believe that's supposed to indicate our defaults?)

Re:disk writing.  I don't disagree, but if we are SRUing it we need to
consider that more.  For 18.04 we can still decide to remove rsyslog to
reduce the impact, we can't do that for 17.10/16.04.

-- 
You received this bug notification because you are a member of Ubuntu
Foundations Bugs, which is subscribed to systemd in Ubuntu.
https://bugs.launchpad.net/bugs/1618188

Title:
  systemd journal should be persistent by default: /var/log/journal
  should be created

Status in systemd package in Ubuntu:
  Fix Released
Status in systemd source package in Xenial:
  In Progress
Status in systemd source package in Zesty:
  Won't Fix
Status in systemd source package in Artful:
  Fix Committed
Status in systemd source package in Bionic:
  Fix Released

Bug description:
  [Impact]

   * System logs are lost across reboots because they are not stored
  persistently.

  [Test Case]

   * Fresh installations, or upgrades to this version of systemd, should create /var/log/journal and trigger automatic persistent logs.
   * Users may choose to remove said directory, or disable persistent logging in /etc/systemd/journald.conf

  [Regression Potential]

   * Persistent logging by default will cause logs to be flushed from
  /run to /var/log, meaning there will be less RAM used (/run is tmpfs
  backed), but increased disk usage (in /var/log). The journald daemon
  has limits set for logs, meaning they will be rotated and discarded
  and should not cause out of disk-space errors.

  [Other Info]
   
   * Original bug report

  After upgrading 14.04 -> 16.04, key services are now running on
  systemd and using the systemd journal for logging. In 14.04, key
  system logs like /var/log/messages and /var/log/syslog were
  persistent, but after the upgrade to 16.04 there has a been a
  regression of sorts: Logs sent to systemd's journald are now being
  thrown away during reboots.

  This behavior is controlled by the `Storage=` option in
  `/etc/systemd/journald.conf`. The default setting is `Storage=auto`
  which will persist logs in `/var/log/journal/`, *only if the directory
  already exists*. But the directory was not created as part of the
  14.04 -> 16.04 upgrade, so logging was being lost for a while before I
  realized what was happening.

  This issue could be solved by either creating /var/log/journal or
  changing the default Storage behavior to `Storage=persistent`, which
  would create the directory if need be.

  ## Related reference

   * `systemd` currently compounds the issue by having ["journal --disk-usage" report memory usage as disk usage](https://github.com/systemd/systemd/issues/4059), giving the impression that the disk is being used for logging when it isn't.
   * [User wonders where to find logs from previous boots, unaware that the logs were thrown away](http://askubuntu.com/questions/765315/how-to-find-previous-boot-log-after-ubuntu-16-04-restarts)

  ## Recommended fix

  Restoring persistent logging as the default is recommended.

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/systemd/+bug/1618188/+subscriptions