[Bug 1934221] Re: systemd-resolve segfault

Fri Sep 17 14:51:37 UTC 2021

Unfortunately, this patch does not seem to be sufficient.

It produces very similar segfaults in on_query_timeout
(https://errors.ubuntu.com/problem/c4e5be3f1c7af9483993c7e6007b9325ab5b78cd)
that the segfault we could observe before
(https://errors.ubuntu.com/problem/bb0ce4ff8e6ef86041cfb5647b792823a20b44f7)

We want to revert the patch in Focal (systemd v245) for now (LP: #1943982), while we'll try to fix the root cause in Hirsute (systemd v247), adding more relevant patches that were added to systemd-stable v247 (but not systemd-stable v245):
https://github.com/systemd/systemd-stable/commits/v247-stable/src/resolve/resolved-dns-query.c
https://github.com/systemd/systemd-stable/commit/64317106aed94a6fb758ab6b08ba490873fc5227
https://github.com/systemd/systemd-stable/commit/91ba2eac4b6b463026b3a93e5a139923e8f2cfe4
https://github.com/systemd/systemd-stable/commit/ab9f7e1a51005f12d3bac83b86716d9d33048eb7
https://github.com/systemd/systemd-stable/commit/78a43c33c8847ebbc2d3cf530ebe304924c58b32
https://github.com/systemd/systemd-stable/commit/c8d7fab2286384b19ff6328cece107c4c02d7bb8

It root-causing is successful, we might back-port those patches to
Focal, too.

** Changed in: systemd (Ubuntu Focal)
       Status: Fix Released => Confirmed

** Changed in: systemd (Ubuntu Hirsute)
       Status: Fix Released => Confirmed

-- 
You received this bug notification because you are a member of Ubuntu
Foundations Bugs, which is subscribed to systemd in Ubuntu.
https://bugs.launchpad.net/bugs/1934221

Title:
  systemd-resolve segfault

Status in systemd package in Ubuntu:
  Fix Released
Status in systemd source package in Focal:
  Confirmed
Status in systemd source package in Hirsute:
  Confirmed
Status in systemd source package in Impish:
  Fix Released

Bug description:
  [Impact]

   * systemd-resolved stops replying to clients on the local LAN.
   * logging segfault crashes in dmesg:
  [836786.046514] systemd-resolve[872009]: segfault at 39900000000 ip 0000039900000000 sp 00007ffd7959a6d8 error 14 in systemd-resolved[556398695000+9000]
  [836786.046524] Code: Bad RIP value.
  [840887.303994] traps: systemd-resolve[877019] general protection fault ip:55ba402e2594 sp:7ffe8cb6bbb0 error:0 in systemd-resolved[55ba402b5000+40000]
  [844395.313421] systemd-resolve[878503]: segfault at 208 ip 00005557a249f5fa sp 00007ffe686f5a90 error 6 in systemd-resolved[5557a2472000+40000]
  [844395.313431] Code: 48 85 c0 74 0e 48 8b 8d 00 01 00 00 48 89 88 00 01 00 00 48 8b 85 00 01 00 00 48 85 c0 0f 84 1d 01 00 00 48 8b 95 f8 00 00 00 <48> 89 90 f8 00 00 00 48 c7 85 00 01 00 00 00 00 00 00 48 c7 85 f8

   * The upload backports the upstream fix
  (https://github.com/systemd/systemd/pull/18832) to Focal & Hirsute.

  [Test Plan]

   * Setup /etc/systemd/resolved.conf:
  [Resolve]
  DNS=46.182.19.48#dns2.digitalcourage.de 1.1.1.1#cloudflare-dns.com 9.9.9.9#dns.quad9.net
  DNSSEC=yes
  DNSOverTLS=opportunistic
  MulticastDNS=no
  LLMNR=no
  Cache=yes
  DNSStubListener=yes
  Domains=~.

   * wait for ~24-48 hours and observe if any crash happens

  [Where problems could occur]

   * Any regression would likely cause crashes in systemd-resolved,
  making it unresponsive to DNS network name requests to local
  applications.

  [Other Info]

   * Reported upstream: https://github.com/systemd/systemd/issues/18427
   * Fixed upstream in v248: https://github.com/systemd/systemd/pull/18832

  === Original description ===

  systemd-resolve keep crashing and it is very annoying as sometimes it severely interrupt normal dns resolving.

  Last uploaded report is 2d9e7378-d89b-11eb-9e14-fa163ee63de6
  Typical error in dmesg:
  systemd-resolve[1792202]: segfault at 564ff982f3e0 ip 0000564ff982f3e0 sp 00007ffe2fd0b758 error 15

  apport hints me that problem is related to mdns

  #3  0x00007f3e903c2f11 in sd_event_dispatch () from
  /lib/systemd/libsystemd-shared-245.so

  It might be (or not) related that some hosts with mdns in my network
  have ipv6 enabled.

  ProblemType: Bug
  DistroRelease: Ubuntu 20.04
  Package: systemd 245.4-4ubuntu3.7
  ProcVersionSignature: Ubuntu 5.4.0-75.84-generic 5.4.119
  Uname: Linux 5.4.0-75-generic x86_64
  ApportVersion: 2.20.11-0ubuntu27.18
  Architecture: amd64
  CasperMD5CheckResult: skip
  CurrentDesktop: ubuntu:GNOME
  Date: Thu Jul  1 08:22:51 2021
  InstallationDate: Installed on 2018-12-05 (938 days ago)
  InstallationMedia: Ubuntu 18.10 "Cosmic Cuttlefish" - Release amd64 (20181017.3)
  MachineType: System manufacturer System Product Name
  ProcKernelCmdLine: BOOT_IMAGE=/boot/vmlinuz-5.4.0-75-generic root=UUID=54b80d5c-3d61-4919-873f-0d308083e3b9 ro quiet splash vt.handoff=7
  SourcePackage: systemd
  UpgradeStatus: Upgraded to focal on 2020-05-22 (405 days ago)
  dmi.bios.date: 05/12/2020
  dmi.bios.vendor: American Megatrends Inc.
  dmi.bios.version: 1205
  dmi.board.asset.tag: Default string
  dmi.board.name: ROG STRIX X399-E GAMING
  dmi.board.vendor: ASUSTeK COMPUTER INC.
  dmi.board.version: Rev 1.xx
  dmi.chassis.asset.tag: Default string
  dmi.chassis.type: 3
  dmi.chassis.vendor: Default string
  dmi.chassis.version: Default string
  dmi.modalias: dmi:bvnAmericanMegatrendsInc.:bvr1205:bd05/12/2020:svnSystemmanufacturer:pnSystemProductName:pvrSystemVersion:rvnASUSTeKCOMPUTERINC.:rnROGSTRIXX399-EGAMING:rvrRev1.xx:cvnDefaultstring:ct3:cvrDefaultstring:
  dmi.product.family: To be filled by O.E.M.
  dmi.product.name: System Product Name
  dmi.product.sku: SKU
  dmi.product.version: System Version
  dmi.sys.vendor: System manufacturer

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/systemd/+bug/1934221/+subscriptions