[Bug 1872118] Re: DHCP Cluster crashes after a few hours

Thu Aug 6 07:28:40 UTC 2020

Not sure if this helps i coped the coredump file to a test system install gdb
and got the following
# gdb /usr/sbin/dhcpd ./CoreDump
GNU gdb (Ubuntu 9.1-0ubuntu1) 9.1
Copyright (C) 2020 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
Type "show copying" and "show warranty" for details.
This GDB was configured as "x86_64-linux-gnu".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
    <http://www.gnu.org/software/gdb/documentation/>.

For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from /usr/sbin/dhcpd...
(No debugging symbols found in /usr/sbin/dhcpd)
[New LWP 63464]
[New LWP 63463]
[New LWP 63448]
[New LWP 63465]

warning: Unexpected size of section `.reg-xstate/63464' in core file.
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
Core was generated by `dhcpd -user dhcpd -group dhcpd -f -4 -pf /run/dhcp-server/dhcpd.pid -cf /etc/dh'.
Program terminated with signal SIGABRT, Aborted.

warning: Unexpected size of section `.reg-xstate/63464' in core file.
#0  __GI_raise (sig=sig at entry=6) at ../sysdeps/unix/sysv/linux/raise.c:50
50      ../sysdeps/unix/sysv/linux/raise.c: No such file or directory.
[Current thread is 1 (Thread 0x7f0ed5c48700 (LWP 63464))]

-- 
You received this bug notification because you are a member of Ubuntu
Foundations Bugs, which is subscribed to isc-dhcp in Ubuntu.
https://bugs.launchpad.net/bugs/1872118

Title:
  DHCP Cluster crashes after a few hours

Status in DHCP:
  New
Status in bind9-libs package in Ubuntu:
  New
Status in isc-dhcp package in Ubuntu:
  Confirmed
Status in bind9-libs source package in Focal:
  New
Status in isc-dhcp source package in Focal:
  New
Status in bind9-libs source package in Groovy:
  New
Status in isc-dhcp source package in Groovy:
  Confirmed

Bug description:

  I have a pair of DHCP serevrs running in a cluster on ubuntu 20.04, All worked perfectly until recently, when they started stopping with code=killed, status=6/ABRT.
  This is being fixed by 

  https://bugs.launchpad.net/bugs/1870729

  However now one stops after a few hours with the following errors. One
  can stay on line but not both.

  Syslog shows 
  Apr 10 17:20:15 dhcp-primary sh[6828]: ../../../../lib/isc/unix/socket.c:3361: INSIST(!sock->pending_send) failed, back trace
  Apr 10 17:20:15 dhcp-primary sh[6828]: #0 0x7fbe78702a4a in ??
  Apr 10 17:20:15 dhcp-primary sh[6828]: #1 0x7fbe78702980 in ??
  Apr 10 17:20:15 dhcp-primary sh[6828]: #2 0x7fbe7873e7e1 in ??
  Apr 10 17:20:15 dhcp-primary sh[6828]: #3 0x7fbe784e5609 in ??
  Apr 10 17:20:15 dhcp-primary sh[6828]: #4 0x7fbe78621103 in ??

  nothing in kern.log

  apport.log shows
  ERROR: apport (pid 6850) Fri Apr 10 17:20:15 2020: called for pid 6828, signal 6, core limit 0, dump mode 2
  ERROR: apport (pid 6850) Fri Apr 10 17:20:15 2020: not creating core for pid with dump mode of 2
  ERROR: apport (pid 6850) Fri Apr 10 17:20:15 2020: executable: /usr/sbin/dhcpd (command line "dhcpd -user dhcpd -group dhcpd -f -4 -pf /run/dhcp-server/dhcpd.pid -cf /etc/dhcp/dhcpd.conf")
  ERROR: apport (pid 6850) Fri Apr 10 17:20:15 2020: is_closing_session(): no DBUS_SESSION_BUS_ADDRESS in environment
  ERROR: apport (pid 6850) Fri Apr 10 17:20:15 2020: wrote report /var/crash/_usr_sbin_dhcpd.0.crash

  /var/crash/_usr_sbin_dhcpd.0.crash shows

  ProblemType: Crash
  Architecture: amd64
  CrashCounter: 1
  Date: Fri Apr 10 17:20:15 2020
  DistroRelease: Ubuntu 20.04
  ExecutablePath: /usr/sbin/dhcpd
  ExecutableTimestamp: 1586210315
  ProcCmdline: dhcpd -user dhcpd -group dhcpd -f -4 -pf /run/dhcp-server/dhcpd.pid -cf /etc/dhcp/dhcpd.conf
  ProcEnviron: Error: [Errno 13] Permission denied: 'environ'
  ProcMaps: Error: [Errno 13] Permission denied: 'maps'
  ProcStatus:
   Name:  dhcpd
   Umask: 0022
   State: D (disk sleep)
   Tgid:  6828
   Ngid:  0
   Pid:   6828
   PPid:  1
   TracerPid:     0
   Uid:   113     113     113     113
   Gid:   118     118     118     118
   FDSize:        128
   Groups:
   NStgid:        6828
   NSpid: 6828
   NSpgid:        6828
   NSsid: 6828
   VmPeak:          236244 kB
   VmSize:          170764 kB
   VmLck:        0 kB
   VmPin:        0 kB
   VmHWM:    12064 kB
   VmRSS:    12064 kB
   RssAnon:           5940 kB
   RssFile:           6124 kB
   RssShmem:             0 kB
   VmData:           30792 kB
   VmStk:      132 kB
   VmExe:      592 kB
   VmLib:     5424 kB
   VmPTE:       76 kB
   VmSwap:               0 kB
   HugetlbPages:         0 kB
   CoreDumping:   1
   THP_enabled:   1
   Threads:       4
   SigQ:  0/7609
   SigPnd:        0000000000000000
   ShdPnd:        0000000000000000
   SigBlk:        0000000000000000
   SigIgn:        0000000000001000
   SigCgt:        0000000180000000
   CapInh:        0000000000000000
   CapPrm:        0000000000000000
   CapEff:        0000000000000000
   CapBnd:        0000003fffffffff
   CapAmb:        0000000000000000
   NoNewPrivs:    0
   Seccomp:       0
   Speculation_Store_Bypass:      thread vulnerable
   Cpus_allowed:  3
   Cpus_allowed_list:     0-1
   Mems_allowed:  00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,0000
  0000,00000000,00000000,00000001
   Mems_allowed_list:     0
   voluntary_ctxt_switches:       111
   nonvoluntary_ctxt_switches:    144
  Signal: 6
  Uname: Linux 5.4.0-21-generic x86_64
  UserGroups:

To manage notifications about this bug go to:
https://bugs.launchpad.net/dhcp/+bug/1872118/+subscriptions