[Bug 1891215] Re: systemd-resolved re-creates /run/systemd/resolve/*resolv.conf for every IPv6 RA received

[Samy Mahmoudi]

Hello,

Some days ago, I installed systemd-dbgsym_245.4-4ubuntu3.6_amd64.ddeb
and set a breakpoint on function
'conservative_renameat':https://github.com/systemd/systemd/blob/main/src/basic
/fs-util.c#L1504 (because of my understanding of function
'manager_write_resolv_conf':https://github.com/systemd/systemd/blob/main/src/resolve
/resolved-resolv-conf.c#L324) to see what happens but I was unsuccessful
catching anything.

Besides, the issue has been reported upstream:
https://github.com/systemd/systemd/issues/17577. Unfortunately, it was
closed after someone suggested that a fix in
https://github.com/systemd/systemd/pull/17535 resolved the systemd-
resolved part of the issue
(https://github.com/systemd/systemd/issues/17577#issuecomment-733962476),
while in truth the combination of
https://github.com/systemd/systemd/pull/17535 and its networkd
counterpart did not solve it.


** Bug watch added: github.com/systemd/systemd/issues #17577
   https://github.com/systemd/systemd/issues/17577

-- 
You received this bug notification because you are a member of Ubuntu
Foundations Bugs, which is subscribed to systemd in Ubuntu.
https://bugs.launchpad.net/bugs/1891215

Title:
  systemd-resolved re-creates /run/systemd/resolve/*resolv.conf for
  every IPv6 RA received

Status in systemd package in Ubuntu:
  Incomplete

Bug description:
  # Issue description:

  On 2 Linode VMs that are used as lxd hosts, we noticed that
  /run/systemd/resolve/*resolv.conf were re-created quite frequently (~
  once per second). We noticed because of the log noise from lxd's
  dnsmasq instance using inotify to watch the target of /etc/resolv.conf
  (which points to the stub-resolv.conf in our case). This was (wrongly)
  reported as a lxd bug (https://github.com/lxc/lxd/issues/7765) until
  it became apparent it was more likely to be a problem with
  systemd(-resolved)?.

  The log noise is the observable problem that would be nice to see
  addressed:

    root at lxd02:~# uptime
     17:55:48 up  9:52,  1 user,  load average: 0.18, 0.11, 0.05
    root at lxd02:~# journalctl -b0 | grep -cF dnsmasq
    158609

  Upon further investigation, it seems that systemd-resolved re-creates
  the resolv.conf and stub-resolv.conf files whenever an IPv6 RA is
  received.

  1) One can observe that by setting systemd-resolved's service in debug
  mode:

  $ sudo systemctl edit systemd-resolved

  and in the editor that is opened, add and save this content:

  [Service]
  Environment=SYSTEMD_LOG_LEVEL=debug

  then restart systemd-resolved and watch the logs scroll by with:

  $ journalctl -fu systemd-resolved

  3) In another terminal, watch the files be recreated with:

  watch -d -n 0.1 stat /run/systemd/resolve/stub-resolv.conf

  3) In yet another terminal, run a packet capture and watch "ICMP6,
  router advertisement" messages come by:

  sudo tcpdump -ni eth0 icmp6

  
  You will see that every time a RA packet comes in, resolved's journal will log this:

    Aug 11 17:33:55 lxd02 systemd-resolved[15368]: Sent message
  type=signal sender=n/a destination=n/a path=/org/freedesktop/resolve1
  interface=org.freedesktop.DBus.Properties member=PropertiesChanged
  cookie=244 reply_cookie=0 signature=sa{sv}as error-name=n/a error-
  message=n/a

  And the stat monitoring terminal will blink to highlight the new inode
  and timestamps of the freshly replaced stub-resolv.conf file.

  
  # Additional information:

  root at lxd02:~# lsb_release -rd
  Description:	Ubuntu 20.04.1 LTS
  Release:	20.04

  root at lxd02:~# apt-cache policy systemd
  systemd:
    Installed: 245.4-4ubuntu3.2
    Candidate: 245.4-4ubuntu3.2
    Version table:
   *** 245.4-4ubuntu3.2 500
          500 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 Packages
          100 /var/lib/dpkg/status
       245.4-4ubuntu3 500
          500 http://archive.ubuntu.com/ubuntu focal/main amd64 Packages

  root at lxd02:~# uname -a
  Linux lxd01 5.4.0-42-generic #46-Ubuntu SMP Fri Jul 10 00:24:02 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/systemd/+bug/1891215/+subscriptions