[Bug 2089565] Re: MON and MDS crash upgrading CEPH on ubuntu 24.04 LTS

[stephane maubian]

Greetings, unrortunnaly for now solution is to put the release on hold until    a new apt package is released, except you want to build from sources
  



  
  
  
  
>   
> Le 25 janv. 2025 à 14:25, Jan Evert van Grootheest  <2089565 at bugs.launchpad.net>  a écrit :
>   
>   
>  May I ask, how to recover from this situation?
>
> I.e. currently I have a cluster using  19.2.0~git20240301.4c76c50-0ubuntu6.
> The latest version as of today is  19.2.0-0ubuntu0.24.04.2.
>
> I've read through the ceph bugreport [1] and a related commit [2]. Also
> the discussion following [3] appears related.
>
> It just doesn't help me find a way forward. If I understand correctly,
> there is not even a way forward.
>
> Any suggestions would be appreciated.
>
>
> [1]  https://tracker.ceph.com/issues/64440
> [2]  https://github.com/ceph/ceph/commit/78abfeaff27fee343fb664db633de5b221699a73
> [3]  https://github.com/ceph/ceph/pull/53340#discussion_r1490282785
>
> ** Bug watch added:  tracker.ceph.com/issues  #64440
>   http://tracker.ceph.com/issues/64440
>
> --  
> You received this bug notification because you are subscribed to the bug
> report.
> https://bugs.launchpad.net/bugs/2089565
>
> Title:
>  MON and MDS crash upgrading CEPH on ubuntu 24.04 LTS
>
> Status in ceph package in Ubuntu:
>  Confirmed
>
> Bug description:
>  This issue is a continuation of
>   https://bugs.launchpad.net/ubuntu/+source/ceph/+bug/2065515
>
>   
>  On Ubuntu 24.04 lts we did upgrade Ceph to  19.2.0-0ubuntu0.24.04.1
>
>  Previous release is :  19.2.0~git20240301.4c76c50-0ubuntu6
>
>  whenever upgrading (tested on 2 different clusters) the ceph-mon
>  ends up crashing repeatedly with the below stack error
>
>  ```
>  ceph version  19.2.0  (16063ff2022298c9300e49a547a16ffda59baf13) squid (stable)
>  1: /lib/x86_64-linux-gnu/libc.so.6(+0x45320) [0x788409245320]
>  2: pthread_kill()
>  3: gsignal()
>  4: abort()
>  5: /lib/x86_64-linux-gnu/libstdc++.so.6(+0xa5ff5) [0x7884096a5ff5]
>  6: /lib/x86_64-linux-gnu/libstdc++.so.6(+0xbb0da) [0x7884096bb0da]
>  7: (std::unexpected()+0) [0x7884096a5a55]
>  8: /lib/x86_64-linux-gnu/libstdc++.so.6(+0xbb391) [0x7884096bb391]
>  9: (ceph::buffer::v15_2_0::list::iterator_impl<true>::copy(unsigned int, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>   >&)+0x193) [0x78840a293593]
>  10: (MDSMap::decode(ceph::buffer::v15_2_0::list::iterator_impl<true>&)+0xca1) [0x78840a4c3ab1]
>  11: (Filesystem::decode(ceph::buffer::v15_2_0::list::iterator_impl<true>&)+0x1c3) [0x78840a4e4303]
>  12: (FSMap::decode(ceph::buffer::v15_2_0::list::iterator_impl<true>&)+0x280) [0x78840a4e6ef0]
>  13: (MDSMonitor::update_from_paxos(bool*)+0x291) [0x631ac5dea801]
>  14: (Monitor::refresh_from_paxos(bool*)+0x124) [0x631ac5b7a164]
>  15: (Monitor::preinit()+0x98e) [0x631ac5bb2fbe]
>  16: main()
>  17: /lib/x86_64-linux-gnu/libc.so.6(+0x2a1ca) [0x78840922a1ca]
>  18: __libc_start_main()
>  19: _start()
>  NOTE: a copy of the executable, or `objdump -rdS  <executable>` is needed to interpret this.
>
>  ```
>
>   
>  mitigation:
>  a rollback to the previous release  19.2.0~git20240301.4c76c50-0ubuntu6 is still possible to restore service
>
> To manage notifications about this bug go to:
> https://bugs.launchpad.net/ubuntu/+source/ceph/+bug/2089565/+subscriptions
>
>

-- 
You received this bug notification because you are a member of Ubuntu
OpenStack, which is subscribed to ceph in Ubuntu.
https://bugs.launchpad.net/bugs/2089565

Title:
  MON and MDS crash upgrading  CEPH  on ubuntu 24.04 LTS

Status in ceph package in Ubuntu:
  Confirmed

Bug description:
  This issue is a continuation of
  https://bugs.launchpad.net/ubuntu/+source/ceph/+bug/2065515

  
  On Ubuntu 24.04 lts we did upgrade Ceph to  19.2.0-0ubuntu0.24.04.1

  Previous release is : 19.2.0~git20240301.4c76c50-0ubuntu6

  whenever  upgrading (tested on 2 different clusters)  the ceph-mon
  ends up crashing repeatedly with the below stack error

  ```
   ceph version 19.2.0 (16063ff2022298c9300e49a547a16ffda59baf13) squid (stable)
   1: /lib/x86_64-linux-gnu/libc.so.6(+0x45320) [0x788409245320]
   2: pthread_kill()
   3: gsignal()
   4: abort()
   5: /lib/x86_64-linux-gnu/libstdc++.so.6(+0xa5ff5) [0x7884096a5ff5]
   6: /lib/x86_64-linux-gnu/libstdc++.so.6(+0xbb0da) [0x7884096bb0da]
   7: (std::unexpected()+0) [0x7884096a5a55]
   8: /lib/x86_64-linux-gnu/libstdc++.so.6(+0xbb391) [0x7884096bb391]
   9: (ceph::buffer::v15_2_0::list::iterator_impl<true>::copy(unsigned int, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >&)+0x193) [0x78840a293593]
   10: (MDSMap::decode(ceph::buffer::v15_2_0::list::iterator_impl<true>&)+0xca1) [0x78840a4c3ab1]
   11: (Filesystem::decode(ceph::buffer::v15_2_0::list::iterator_impl<true>&)+0x1c3) [0x78840a4e4303]
   12: (FSMap::decode(ceph::buffer::v15_2_0::list::iterator_impl<true>&)+0x280) [0x78840a4e6ef0]
   13: (MDSMonitor::update_from_paxos(bool*)+0x291) [0x631ac5dea801]
   14: (Monitor::refresh_from_paxos(bool*)+0x124) [0x631ac5b7a164]
   15: (Monitor::preinit()+0x98e) [0x631ac5bb2fbe]
   16: main()
   17: /lib/x86_64-linux-gnu/libc.so.6(+0x2a1ca) [0x78840922a1ca]
   18: __libc_start_main()
   19: _start()
   NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this.

  ```

  
  mitigation:
  a rollback to the previous release 19.2.0~git20240301.4c76c50-0ubuntu6 is still possible to restore service

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/ceph/+bug/2089565/+subscriptions