[Bug 1886128] Re: systemd-resolved does not resolve address due to udp payload size.

[Dan Streetman]

To repro on azure instance:

ddstreet at lp1886128:~$ systemd-resolve --status | grep Servers
         DNS Servers: 168.63.129.16
ddstreet at lp1886128:~$ dig +retries=0 +timeout=1 +short +tcp @168.63.129.16 toomany100.ddstreet.org
;; connection timed out; no servers could be reached
;; Connection to 168.63.129.16#53(168.63.129.16) for toomany100.ddstreet.org failed: timed out.


tcp does work with external dns servers, e.g.:

ddstreet at lp1886128:~$ dig +retries=0 +timeout=1 +short +tcp @8.8.8.8 toomany100.ddstreet.org | head -5
10.254.201.0
10.254.201.1
10.254.201.18
10.254.201.19
10.254.201.2


note that the TCP packets to 168.63.129.16 do not make it out of the instance; they don't show up in the tcpdump at all, although TCP packets to 168.63.129.15 or 168.63.129.17 *do* show up.  So there is something specific to that ip address that's causing TCP packets to get lost inside the instance.

-- 
You received this bug notification because you are a member of Ubuntu
Foundations Bugs, which is subscribed to systemd in Ubuntu.
https://bugs.launchpad.net/bugs/1886128

Title:
  systemd-resolved does not resolve address due to udp payload size.

Status in systemd package in Ubuntu:
  In Progress

Bug description:
  Description:	Ubuntu 18.04.4 LTS
  Release:	18.04

  systemd-resolve --version

  systemd 237
  +PAM +AUDIT +SELINUX +IMA +APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP
  +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD -IDN2 +IDN
  -PCRE2 default-hierarchy=hybrid

  We met an error: on an attempt to resolve address, the following issue
  appears:

  ; <<>> DiG 9.11.3-1ubuntu1.11-Ubuntu <<>> mharder-formrec.cognitiveservices.azure.com
  ;; global options: +cmd
  ;; Got answer:
  ;; ->>HEADER<<- opcode: QUERY, status: SERVFAIL, id: 44096
  ;; flags: qr rd ra; QUERY: 1, ANSWER: 0, AUTHORITY: 0, ADDITIONAL: 1

  ;; OPT PSEUDOSECTION:
  ; EDNS: version: 0, flags:; udp: 65494
  ;; QUESTION SECTION:
  ;mharder-formrec.cognitiveservices.azure.com. IN	A

  ;; Query time: 231 msec
  ;; SERVER: 127.0.0.53#53(127.0.0.53)
  ;; WHEN: Tue Apr 28 20:47:14 UTC 2020
  ;; MSG SIZE  rcvd: 72

  Let me provide you important notes about the issue:
  1) It's not reproducing on Ubuntu 16;
  2) Bypassing systemd-resolve - everything works fine;
  3) Only the difference between systemd-resolve and END is UDP_PAYLOAD_SIZE

  Successful query:

  1135    16:27:25.964386 10.1.0.4        168.63.129.16   DNS     128
  Standard query 0xc2d4 A mharder-formrec.cognitiveservices.azure.com
  OPT

  Domain Name System (query)
      Transaction ID: 0xc2d4
      Flags: 0x0120 Standard query
          0... .... .... .... = Response: Message is a query
          .000 0... .... .... = Opcode: Standard query (0)
          .... ..0. .... .... = Truncated: Message is not truncated
          .... ...1 .... .... = Recursion desired: Do query recursively
          .... .... .0.. .... = Z: reserved (0)
          .... .... ..1. .... = AD bit: Set
          .... .... ...0 .... = Non-authenticated data: Unacceptable
      Questions: 1
      Answer RRs: 0
      Authority RRs: 0
      Additional RRs: 1
      Queries
          mharder-formrec.cognitiveservices.azure.com: type A, class IN
      Additional records
          <Root>: type OPT
              Name: <Root>
              Type: OPT (41)
              UDP payload size: 4096
              Higher bits in extended RCODE: 0x00
              EDNS0 version: 0
              Z: 0x0000
                  0... .... .... .... = DO bit: Cannot handle DNSSEC security RRs
                  .000 0000 0000 0000 = Reserved: 0x0000
              Data length: 12
              Option: COOKIE
  Unsuccessful query:

  1128    16:27:25.713886 10.1.0.4        168.63.129.16   DNS     116
  Standard query 0x198d A mharder-formrec.cognitiveservices.azure.com
  OPT

  Domain Name System (query)
      Transaction ID: 0x198d
      Flags: 0x0100 Standard query
          0... .... .... .... = Response: Message is a query
          .000 0... .... .... = Opcode: Standard query (0)
          .... ..0. .... .... = Truncated: Message is not truncated
          .... ...1 .... .... = Recursion desired: Do query recursively
          .... .... .0.. .... = Z: reserved (0)
          .... .... ...0 .... = Non-authenticated data: Unacceptable
      Questions: 1
      Answer RRs: 0
      Authority RRs: 0
      Additional RRs: 1
      Queries
          mharder-formrec.cognitiveservices.azure.com: type A, class IN
      Additional records
          <Root>: type OPT
              Name: <Root>
              Type: OPT (41)
              UDP payload size: 512
              Higher bits in extended RCODE: 0x00
              EDNS0 version: 0
              Z: 0x0000
                  0... .... .... .... = DO bit: Cannot handle DNSSEC security RRs
                  .000 0000 0000 0000 = Reserved: 0x0000
              Data length: 0
  Notable difference:

  Success:
              UDP payload size: 4096

  Failure:
              UDP payload size: 512
  And notable differences in the responses:

  Success:
      Flags: 0x8180 Standard query response, No error
          .... ..0. .... .... = Truncated: Message is not truncated

  Failure:
      Flags: 0x8380 Standard query response, No error
          .... ..1. .... .... = Truncated: Message is truncated

  Interestingly, systemd-resolved is setting the maximum payload size to 512 regardless of whether EDNS0 is configured and regardless of what is sent to it for the payload size.
  I tried to found a way to change UDP_PAYLOAD_SIZE,but it seems it is only possible to change it only with direct code modifications.

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/systemd/+bug/1886128/+subscriptions