[Bug 1950850] Re: Systemd-resolved gets confused and replies to SOA queries for local breaking mDNS resolution

[Nick Rosbrook]

Please open a new bug if this is still observed on newer releases.

** Changed in: systemd (Ubuntu)
       Status: New => Won't Fix

-- 
You received this bug notification because you are a member of Ubuntu
Foundations Bugs, which is subscribed to systemd in Ubuntu.
https://bugs.launchpad.net/bugs/1950850

Title:
  Systemd-resolved gets confused and replies to SOA queries for local
  breaking mDNS resolution

Status in systemd package in Ubuntu:
  Won't Fix

Bug description:
  As discussed on https://gitlab.nic.cz/knot/knot-resolver/-/issues/686,
  DNS servers are expected to immediately answer any queries within
  local by NXDOMAIN. See
  https://datatracker.ietf.org/doc/html/rfc6762#section-22.1 in this
  respect. However, while they do so they seem to be  also expected to
  deliver an AUTHORITY section with a SOA. As a matter of fact, I am
  being told the relevant RFCs do not explicitly mention that SOA should
  be there, but that NXDOMAIN without a SOA is generally considered
  incorrect, e.g. it necessarily causes caching issues due to missing
  TTL.

  The problem is that such SOA confuses systemd-resolved (at least the
  version distributed with Ubuntu Focal) that ends up converting the
  NXDOMAIN reply into a positive reply with such a SOA.

  The final consequence is that local name resolution via mDNS is broken
  with the default ubuntu focal setup.

  
  A detailed discussion of the matter and of the scenario triggering the issue follows.

  
  Suppose that on your network you have a DNS server following the .local blocking indicated in https://datatracker.ietf.org/doc/html/rfc6762#section-22.1 (bullet 4). An example of a modern DNS that has already implemented this feature is knot resolver and the feature is also present in some routers notably the Turris Omnia series. Such a DNS server would reply to queries to local in this way:

    $ dig local
  ;; WARNING: .local is reserved for Multicast DNS
  ;; You are currently testing what happens when an mDNS query is leaked to DNS
  ;; ->>HEADER<<- opcode: QUERY, status: NXDOMAIN, id: 56352
  ;; flags: qr aa rd ra; QUERY: 1, ANSWER: 0, AUTHORITY: 1, ADDITIONAL: 1

  ;; QUESTION SECTION:

  ;local.                         IN      A

  ;; AUTHORITY SECTION:

  local.                  10800   IN      SOA     local. nobody.invalid.
  1 3600 1200 604800 10800

  ;; ADDITIONAL SECTION:

  explanation.invalid.    10800   IN      TXT     “Blocking is mandated
  by standards, see references on
  https://www.iana.org/assignments/special-use-domain-names/special-use-
  domain-names.xhtml”

  Now let's see what happens to Ubuntu Focal hosts when such a DNS
  server is deployed in the network and you try to reach a local host:

  1.  You do something like ping foo.local or ssh foo.local

  2.  Ubuntu focal has by default the host field in nsswitch.conf set
  to:

      hosts:          files mdns4_minimal [NOTFOUND=return] dns so it
  tries the /etc/hosts/ file and then mdns via the nss mdns4_minimal
  client

  3.  The mdns4_minimal client before doing anything else tries unicast DNS looking for a SOA for
      local. This mechanism is present in the mdns4_minimal client to avoid issues when local is
      under DNS control and is documented at

      https://github.com/lathiat/nss-mdns/blob/master/README.md
      
  4.  Ubuntu focal uses by default systemd-resolved as a caching DNS, so the query from
      mdns4_minimal gets to systemd-resolved

  5. systemd-resolved passes the query to the DNS it is configured to use. If this is a modern DNS
     server blocking local it gets NXDOMAIN together withthat special SOA in the authority section
     and because it is buggy, it turns it into a regular SOA reply (no NXDOMAIN)

  6. mdns4_minimal receives a SOA reply for local and gives up thinking that in the particular
     environment local is under unicast DNS control

  7. At this point, according to the nsswitch configuration, DNS is tried. So we are back to
     systemd-resolved now trying to get the A field for foo.local.

  8. By default systemd-resolved on ubuntu is configured not to do mDNS itself (even if it has 
     this capability). Hence the query fails.

  9. Rather than reaching foo.local you get an error.

  
  This is a nasty but, although it is possibly not frequently hit yet because the number of DNS servers that have already implemented local blocking is probably modest. You can only expect it to be on the rise, though.

  
  It may be the case that more recent versions of systemd than the one shipped in Ubuntu Focal have already corrected the issue. Preliminary tests on arch seem to support this hypothesis. If this is the case, it may be possible to backport the fix.

  
  In any case, I think that an update to the Ubuntu Focal systemd package should be provided, because Focal deployments are expected to exist for many years ahead.

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/systemd/+bug/1950850/+subscriptions