[Bug 2157583] Re: NFS-Ganesha FSAL_CEPH AddExport segfaults in libcephfs

[Luciano Lo Giudice]

The full backtrace, with debug symbols enabled:

#0  0x00007cf71aa5aff6 in MDSMap::decode (this=this at entry=0x7cf608009230, p=...) at /usr/src/ceph-20.2.0-0ubuntu2/src/mds/MDSMap.cc:1063
#1  0x00007cf718943c0f in MDSMap::decode (this=0x7cf608009230, bl=...) at /usr/src/ceph-20.2.0-0ubuntu2/src/mds/MDSMap.h:568
#2  Client::handle_mds_map (this=0x7cf65cc00e80, m=...) at /usr/src/ceph-20.2.0-0ubuntu2/src/client/Client.cc:3228
#3  0x00007cf718944849 in Client::ms_dispatch2 (this=0x7cf65cc00e80, m=...) at /usr/src/ceph-20.2.0-0ubuntu2/src/client/Client.cc:3095
#4  0x00007cf71a6ac2c9 in Messenger::ms_deliver_dispatch (this=0x7cf65cb7bf30, m=...) at /usr/src/ceph-20.2.0-0ubuntu2/src/msg/Messenger.h:746
#5  0x00007cf71a6aa2a2 in DispatchQueue::entry (this=0x7cf65cb7c238) at /usr/src/ceph-20.2.0-0ubuntu2/src/msg/DispatchQueue.cc:202
#6  0x00007cf71a7859a1 in DispatchQueue::DispatchThread::entry (this=<optimized out>) at /usr/src/ceph-20.2.0-0ubuntu2/src/msg/DispatchQueue.h:101
#7  0x00007cf71b4100da in start_thread (arg=<optimized out>) at ./nptl/pthread_create.c:454
#8  0x00007cf71b4a37ac in __GI___clone3 () at ../sysdeps/unix/sysv/linux/x86_64/clone3.S:78

After debugging, the problem was caused by an incomplete patch for
snapshot upgrades. This has been fixed in the PPA: lmlogiudice/ceph-
tentacle-rc and will be available once packages for 20.2.1 are merged.

-- 
You received this bug notification because you are a member of Ubuntu
OpenStack, which is subscribed to ceph in Ubuntu.
https://bugs.launchpad.net/bugs/2157583

Title:
  NFS-Ganesha FSAL_CEPH AddExport segfaults in libcephfs

Status in ceph package in Ubuntu:
  New

Bug description:
  On Ubuntu 26.04 / Resolute with Ceph Tentacle packages, `ganesha.nfsd`
  segfaults when dynamically adding a CephFS export through DBus
  `ExportMgr.AddExport`.

  The crash occurs after FSAL_CEPH starts creating a CephFS mount and
  immediately after the FSAL_CEPH reclaim-reset path logs:

  ```text
  reclaim_reset :FSAL :DEBUG :Issuing reclaim reset for ganesha-...
  ```

  GDB shows the faulting thread is a Ceph messenger dispatch thread
  inside `MDSMap::decode()` from `libceph-common.so.2`, reached via
  `libcephfs.so.2`.

  This blocks the `ceph-nfs` charm on Resolute/Tentacle because the
  `create-share` action writes an export object, calls
  `ExportMgr.AddExport`, and then fails when Ganesha crashes before
  replying.

  ```

  ## Affected versions

  Observed on the reproducing unit:

  ```text
  ceph-common              20.2.0-0ubuntu2
  libcephfs2               20.2.0-0ubuntu2
  nfs-ganesha              6.5-8
  ```

  Environment:

  ```text
  Ubuntu base: ubuntu at 26.04 amd64
  Ceph release: Tentacle / 20.2.0
  ```

  
  GDB backtrace excerpt

  ```text
  Thread 64 "ms_dispatch" received signal SIGSEGV, Segmentation fault.
  0x00007ffff719bff6 in MDSMap::decode(ceph::buffer::v15_2_0::list::iterator_impl<true>&) ()
     from /usr/lib/x86_64-linux-gnu/ceph/libceph-common.so.2

  #0  0x00007ffff719bff6 in MDSMap::decode(ceph::buffer::v15_2_0::list::iterator_impl<true>&) ()
      from /usr/lib/x86_64-linux-gnu/ceph/libceph-common.so.2
  #1  0x00007ffff51e0c0f in ?? () from /usr/lib/x86_64-linux-gnu/libcephfs.so.2
  #2  0x00007ffff51e1849 in ?? () from /usr/lib/x86_64-linux-gnu/libcephfs.so.2
  #3  0x00007ffff6ded2c9 in Messenger::ms_deliver_dispatch(boost::intrusive_ptr<Message> const&) ()
      from /usr/lib/x86_64-linux-gnu/ceph/libceph-common.so.2
  #4  0x00007ffff6deb2a2 in DispatchQueue::entry() ()
      from /usr/lib/x86_64-linux-gnu/ceph/libceph-common.so.2
  #5  0x00007ffff6ec69a1 in DispatchQueue::DispatchThread::entry() ()
      from /usr/lib/x86_64-linux-gnu/ceph/libceph-common.so.2
  #6  0x00007ffff7aa40da in start_thread (arg=<optimized out>)
  #7  0x00007ffff7b377ac in __GI___clone3 ()
  ```

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/ceph/+bug/2157583/+subscriptions