[Bug 1732803] Re: systemd-journald RateLimitBurst is sometimes divided by 4

[David Glasser]

This is my first attempt at a debdiff and an SRU. I'd love to know if
I've filed it properly!

-- 
You received this bug notification because you are a member of Ubuntu
Sponsors Team, which is subscribed to the bug report.
https://bugs.launchpad.net/bugs/1732803

Title:
  systemd-journald RateLimitBurst is sometimes divided by 4

Status in systemd package in Ubuntu:
  New

Bug description:
  [Impact]

  systemd-journald allows you to configure a per-service journal rate limit in /etc/systemd/journald.conf via the RateLimitBurst parameter. systemd-journald has
  code that effectively increases the rate limit when there is a lot of disk space available.
  However, all versions of systemd before v232 had a bug which would shrink the rate limit
  when there is between 1 and 16 MB available on disk.

  If you designed a service to log at a rate R and configured
  RateLimitBurst to a little above R, this can lead to loss of logs when
  free disk is between 1 and 16 MB, as your service will be surprisingly
  rate limited at lower than your configured rate.

  This bug was fixed upstream in https://github.com/systemd/systemd/pull/4218
  It is a straightforward one-line change that makes the code match the comments under it.

  [Test Case]

  Run a systemd service that prints lots of logs (eg `yes`). Fill your
  disk to have only 1MB full. Use journalctl to see how many log lines
  are between "Suppressed" lines. Note that it is 1/4 of what you'd
  expect.  (Admittedly this test case is a little hard to achieve since
  journald itself is writing to disk. I did run into this in
  production.)

  [Regression Potential]

  This does mean that journald can write slightly more to disk than it
  did before when free disk is between 1 and 16MB, but given that the
  full burst rate is available below 1MB it seems unlikely that any
  systems are depending on this change in order to not break.

  The fix has been in systemd since v232 (shipped in Zesty). I would
  like to see it in Xenial.

  
  [Other Info]
   
  I am seeing this on:
  ubuntu at ip-10-0-2-135[i-0b196ce4b8dc3fc55] 1 ~/systemd-229$ lsb_release -rd
  Description:	Ubuntu 16.04.3 LTS
  Release:	16.04
  ubuntu at ip-10-0-2-135[i-0b196ce4b8dc3fc55] 0 ~/systemd-229$ apt-cache policy systemd
  systemd:
    Installed: 229-4ubuntu19
    Candidate: 229-4ubuntu21
    Version table:
       229-4ubuntu21 500
          500 http://us-east-1.ec2.archive.ubuntu.com/ubuntu xenial-updates/main amd64 Packages
   *** 229-4ubuntu19 100
          100 /var/lib/dpkg/status
       229-4ubuntu10 500
          500 http://security.ubuntu.com/ubuntu xenial-security/main amd64 Packages
       229-4ubuntu4 500
          500 http://us-east-1.ec2.archive.ubuntu.com/ubuntu xenial/main amd64 Packages

  This bug seems to date back to the original implementation of rate
  limiting (https://github.com/systemd/systemd/commit/6e409ce10d).

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/systemd/+bug/1732803/+subscriptions