[Bug 1805145] Re: radosgw crashed with SIGABRT in __GI_raise() when uploading small file via Horizon/CLI

Chris MacNaughton 1805145 at bugs.launchpad.net
Tue Dec 11 16:23:19 UTC 2018


I can confirm that radosgw works as expected when deploying the cloud
from rocky-proposed

-- 
You received this bug notification because you are a member of Ubuntu
OpenStack, which is subscribed to Ubuntu Cloud Archive.
https://bugs.launchpad.net/bugs/1805145

Title:
  radosgw crashed with SIGABRT in __GI_raise() when uploading small file
  via Horizon/CLI

Status in Ubuntu Cloud Archive:
  Fix Released
Status in Ubuntu Cloud Archive rocky series:
  Fix Released
Status in Ubuntu Cloud Archive stein series:
  Fix Released
Status in ceph package in Ubuntu:
  Fix Released
Status in ceph source package in Cosmic:
  In Progress
Status in ceph source package in Disco:
  Fix Released

Bug description:
  [Impact]
  radosgw is non functional - files cannot be uploaded.

  [Test Case]
  Deploy ceph + radosgw
  Upload file using swift cli tooling
  radosgw daemon SIGABRT's.

  [Regression Potential]
  The cause of this bug is a distro patch for 32bit compatibility which mis-aligns the type for the retrieval of a configuration option with the underlying type of the option - this results in a SIGABRT.

  The update to the distro patch realigns the types and aligns with a
  broader patch in upstream ceph in master branch development.

  [Original Bug Report]
  Hi,

  Juju deployed OpenStack using a modified openstack-core bundle - Rocky
  on bionic.  All components (except neutron-gateway) are deployed in
  LXD containers.

  Horizon/CLI can list and create new containers, however, if a small
  file (in this case a cloud.init file of 82 bytes) is uploaded, the
  radosgw process crashes with attached crash file.

  NB this is running within GCP

  Please advise if any further information required.  Here's the snippet
  out of the end of radosgw.log

  d9fa6cacbb94b96b4941ec2af38a49f$ad9fa6cacbb94b96b4941ec2af38a49f:head [getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected e80) v8 -- 0x5587e73f9740 con 0
     -27> 2018-11-26 12:17:31.262 7f87d596b700  5 -- 252.0.89.179:0/1413084365 >> 252.0.80.134:6800/7301 conn(0x5587e6c51800 :-1 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=20 cs=1 l=1). rx osd.0 seq 10832 0x5587e73f9740 osd_op_reply(132197 ad9fa6cacbb94b96b4941ec2af38a49f$ad9fa6cacbb94b96b4941ec2af38a49f [getxattrs,stat] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8
     -26> 2018-11-26 12:17:31.262 7f87d596b700  1 -- 252.0.89.179:0/1413084365 <== osd.0 252.0.80.134:6800/7301 10832 ==== osd_op_reply(132197 ad9fa6cacbb94b96b4941ec2af38a49f$ad9fa6cacbb94b96b4941ec2af38a49f [getxattrs,stat] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 251+0+0 (2451902153 0 0) 0x5587e73f9740 con 0x5587e6c51800
     -25> 2018-11-26 12:17:31.262 7f87b633a700  2 req 23:0.001665:swift:PUT /swift/v1/test/cloudinit.txt:put_obj:normalizing buckets and tenants
     -24> 2018-11-26 12:17:31.262 7f87b633a700  2 req 23:0.001701:swift:PUT /swift/v1/test/cloudinit.txt:put_obj:init permissions
     -23> 2018-11-26 12:17:31.262 7f87b633a700  2 req 23:0.001755:swift:PUT /swift/v1/test/cloudinit.txt:put_obj:recalculating target
     -22> 2018-11-26 12:17:31.262 7f87b633a700  2 req 23:0.001779:swift:PUT /swift/v1/test/cloudinit.txt:put_obj:reading permissions
     -21> 2018-11-26 12:17:31.262 7f87b633a700  2 req 23:0.001786:swift:PUT /swift/v1/test/cloudinit.txt:put_obj:init op
     -20> 2018-11-26 12:17:31.262 7f87b633a700  2 req 23:0.001788:swift:PUT /swift/v1/test/cloudinit.txt:put_obj:verifying op mask
     -19> 2018-11-26 12:17:31.262 7f87b633a700  2 req 23:0.001789:swift:PUT /swift/v1/test/cloudinit.txt:put_obj:verifying op permissions
     -18> 2018-11-26 12:17:31.262 7f87b633a700  5 content type wasn't provided, trying to guess
     -17> 2018-11-26 12:17:31.262 7f87b633a700  5 Searching permissions for identity=rgw::auth::ThirdPartyAccountApplier() -> rgw::auth::SysReqApplier -> rgw::auth::RemoteApplier(acct_user=ad9fa6cacbb94b96b4941ec2af38a49f, acct_name=xxxx_mgmt, perm_mask=15, is_admin=0) mask=50
     -16> 2018-11-26 12:17:31.262 7f87b633a700  5 Searching permissions for uid=ad9fa6cacbb94b96b4941ec2af38a49f
     -15> 2018-11-26 12:17:31.262 7f87b633a700  5 Found permission: 15
     -14> 2018-11-26 12:17:31.262 7f87b633a700  5 Searching permissions for uid=ad9fa6cacbb94b96b4941ec2af38a49f$ad9fa6cacbb94b96b4941ec2af38a49f
     -13> 2018-11-26 12:17:31.262 7f87b633a700  5 Permissions for user not found
     -12> 2018-11-26 12:17:31.262 7f87b633a700  5 Searching permissions for group=1 mask=50
     -11> 2018-11-26 12:17:31.262 7f87b633a700  5 Permissions for group not found
     -10> 2018-11-26 12:17:31.262 7f87b633a700  5 Searching permissions for group=2 mask=50
      -9> 2018-11-26 12:17:31.262 7f87b633a700  5 Permissions for group not found
      -8> 2018-11-26 12:17:31.262 7f87b633a700  5 -- Getting permissions done for identity=rgw::auth::ThirdPartyAccountApplier() -> rgw::auth::SysReqApplier -> rgw::auth::RemoteApplier(acct_user=ad9fa6cacbb94b96b4941ec2af38a49f, acct_name=xxxxx_mgmt, perm_mask=15, is_admin=0), owner=ad9fa6cacbb94b96b4941ec2af38a49f, perm=2
      -7> 2018-11-26 12:17:31.262 7f87b633a700  2 req 23:0.001818:swift:PUT /swift/v1/test/cloudinit.txt:put_obj:verifying op params
      -6> 2018-11-26 12:17:31.262 7f87b633a700  2 req 23:0.001820:swift:PUT /swift/v1/test/cloudinit.txt:put_obj:pre-executing
      -5> 2018-11-26 12:17:31.262 7f87b633a700  2 req 23:0.001821:swift:PUT /swift/v1/test/cloudinit.txt:put_obj:executing
      -4> 2018-11-26 12:17:31.262 7f87b633a700  1 -- 252.0.89.179:0/1413084365 --> 252.0.90.230:6800/17596 -- osd_op(unknown.0.0:132198 6.0 6:21b9d9f7:::.dir.c9c6424e-a4bd-44de-b972-6cd4539a0a61.5322.1:head [call rgw.bucket_list] snapc 0=[] ondisk+read+known_if_redirected e80) v8 -- 0x5587e745a9c0 con 0
      -3> 2018-11-26 12:17:31.266 7f87d616c700  5 -- 252.0.89.179:0/1413084365 >> 252.0.90.230:6800/17596 conn(0x5587e6c50300 :-1 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=10 cs=1 l=1). rx osd.1 seq 54313 0x5587e745a9c0 osd_op_reply(132198 .dir.c9c6424e-a4bd-44de-b972-6cd4539a0a61.5322.1 [call] v0'0 uv1 ondisk = 0) v8
      -2> 2018-11-26 12:17:31.266 7f87d616c700  1 -- 252.0.89.179:0/1413084365 <== osd.1 252.0.90.230:6800/17596 54313 ==== osd_op_reply(132198 .dir.c9c6424e-a4bd-44de-b972-6cd4539a0a61.5322.1 [call] v0'0 uv1 ondisk = 0) v8 ==== 192+0+71 (1074935246 0 1943469035) 0x5587e745a9c0 con 0x5587e6c50300
      -1> 2018-11-26 12:17:31.266 7f87b633a700  5 NOTICE: call to do_aws4_auth_completion
       0> 2018-11-26 12:17:31.266 7f87b633a700 -1 *** Caught signal (Aborted) **
   in thread 7f87b633a700 thread_name:civetweb-worker

   ceph version 13.2.1 (5533ecdc0fda920179d7ad84e0aa65a127b20d77) mimic (stable)
   1: (()+0x293e70) [0x5587e4514e70]
   2: (()+0x12890) [0x7f87eb99d890]
   3: (gsignal()+0xc7) [0x7f87e0e96e97]
   4: (abort()+0x141) [0x7f87e0e98801]
   5: (()+0x8c8b7) [0x7f87e14ed8b7]
   6: (()+0x92a06) [0x7f87e14f3a06]
   7: (()+0x92a41) [0x7f87e14f3a41]
   8: (()+0x92c74) [0x7f87e14f3c74]
   9: (()+0x35e30d) [0x5587e45df30d]
   10: (Option::size_t const md_config_t::get_val<Option::size_t>(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) const+0x53) [0x5587e461fab3]
   11: (()+0x37df4f) [0x5587e45fef4f]
   12: (RGWPutObj::execute()+0x1adb) [0x5587e4600efb]
   13: (rgw_process_authenticated(RGWHandler_REST*, RGWOp*&, RGWRequest*, req_state*, bool)+0x1cd) [0x5587e4620cfd]
   14: (process_request(RGWRados*, RGWREST*, RGWRequest*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, rgw::auth::StrategyRegistry const&, RGWRestfulIO*, OpsLogSocket*, int*)+0x15f0) [0x5587e46223f0]
   15: (RGWCivetWebFrontend::process(mg_connection*)+0x381) [0x5587e44976e1]
   16: (()+0x2851ee) [0x5587e45061ee]
   17: (()+0x286e78) [0x5587e4507e78]
   18: (()+0x287308) [0x5587e4508308]
   19: (()+0x76db) [0x7f87eb9926db]
   20: (clone()+0x3f) [0x7f87e0f7988f]
   NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this.

  --- logging levels ---
     0/ 5 none
     0/ 1 lockdep
     0/ 1 context
     1/ 1 crush
     1/ 5 mds
     1/ 5 mds_balancer
     1/ 5 mds_locker
     1/ 5 mds_log
     1/ 5 mds_log_expire
     1/ 5 mds_migrator
     0/ 1 buffer
     0/ 1 timer
     0/ 1 filer
     0/ 1 striper
     0/ 1 objecter
     0/ 5 rados
     0/ 5 rbd
     0/ 5 rbd_mirror
     0/ 5 rbd_replay
     0/ 5 journaler
     0/ 5 objectcacher
     0/ 5 client
     1/ 5 osd
     0/ 5 optracker
     0/ 5 objclass
     1/ 3 filestore
     1/ 3 journal
     0/ 5 ms
     1/ 5 mon
     0/10 monc
     1/ 5 paxos
     0/ 5 tp
     1/ 5 auth
     1/ 5 crypto
     1/ 1 finisher
     1/ 1 reserver
     1/ 5 heartbeatmap
     1/ 5 perfcounter
     1/ 5 rgw
     1/ 5 rgw_sync
     1/10 civetweb
     1/ 5 javaclient
     1/ 5 asok
     1/ 1 throttle
     0/ 0 refs
     1/ 5 xio
     1/ 5 compressor
     1/ 5 bluestore
     1/ 5 bluefs
     1/ 3 bdev
     1/ 5 kstore
     4/ 5 rocksdb
     4/ 5 leveldb
     4/ 5 memdb
     1/ 5 kinetic
     1/ 5 fuse
     1/ 5 mgr
     1/ 5 mgrc
     1/ 5 dpdk
     1/ 5 eventtrace
    -2/-2 (syslog threshold)
    -1/-1 (stderr threshold)
    max_recent     10000
    max_new         1000
    log_file /var/log/ceph/radosgw.log
  --- end dump of recent events ---

  ProblemType: Crash
  DistroRelease: Ubuntu 18.04
  Package: radosgw 13.2.1+dfsg1-0ubuntu2~cloud0 [origin: Canonical]
  ProcVersionSignature: Ubuntu 4.15.0-1024.25-gcp 4.15.18
  Uname: Linux 4.15.0-1024-gcp x86_64
  ApportVersion: 2.20.9-0ubuntu7.5
  Architecture: amd64
  CrashDB:
   {
                  "impl": "launchpad",
                  "project": "cloud-archive",
                  "bug_pattern_url": "http://people.canonical.com/~ubuntu-archive/bugpatterns/bugpatterns.xml",
               }
  Date: Mon Nov 26 12:17:31 2018
  ExecutablePath: /usr/bin/radosgw
  ProcCmdline: /usr/bin/radosgw -n client.radosgw.gateway
  ProcEnviron:
   PATH=(custom, no user)
   LANG=C.UTF-8
  Signal: 6
  SourcePackage: ceph
  StacktraceTop:
   __GI_raise (sig=sig at entry=6) at ../sysdeps/unix/sysv/linux/raise.c:51
   __GI_abort () at abort.c:79
   ?? () from /usr/lib/x86_64-linux-gnu/libstdc++.so.6
   ?? () from /usr/lib/x86_64-linux-gnu/libstdc++.so.6
   std::terminate() () from /usr/lib/x86_64-linux-gnu/libstdc++.so.6
  Title: radosgw crashed with SIGABRT in __GI_raise()
  UpgradeStatus: No upgrade log present (probably fresh install)
  UserGroups:

To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-archive/+bug/1805145/+subscriptions



More information about the Ubuntu-openstack-bugs mailing list