[Bug 1774279] Re: unable to create pools before OSD's are up and running

[James Page]

Raising Ubuntu task; it would be nice if we could tell the ceph-mon
cluster how many OSD's there *will* be, rather than it defaulting to '3'
if there are no OSD's currently checked in.

This will require a new configuration option and adoption by the ceph
project.

For a charm based deployment, the value of 'in' osds is unreliable until
the cloud is fully deployed; having this trip hazard is non-ideal.

-- 
You received this bug notification because you are a member of Ubuntu
OpenStack, which is subscribed to ceph in Ubuntu.
https://bugs.launchpad.net/bugs/1774279

Title:
  unable to create pools before OSD's are up and running

Status in OpenStack ceph-mon charm:
  Triaged
Status in OpenStack ceph-osd charm:
  Triaged
Status in ceph package in Ubuntu:
  New

Bug description:
  When we deploy our Openstack bundle with gnocchi in it, most of the
  time the charms will be in error.

  After looking at the logs, it seems gnocchi tried to create its pool
  in ceph, but it does so after ceph-mon is ready but before ceph-
  radosgw is also ready.

  The logs on ceph-radosgw are filled with the error messages:
  2018-05-30 21:09:25.739133 7f48374f4e80  0 pidfile_write: ignore empty --pid-file
  2018-05-30 21:09:25.748857 7f48374f4e80 -1 auth: unable to find a keyring on /etc/ceph/keyring.rados.gateway: (2) No such file or directory

  The file "/etc/ceph/keyring.rados.gateway" is indeed not present.
  On a deployment that worked, it was present.

  Also on ceph-mon no other pools get to be created, no matter how long we wait.
  # ceph df
  GLOBAL:
      SIZE     AVAIL     RAW USED     %RAW USED 
      488G      488G         325M          0.07 
  POOLS:
      NAME                    ID     USED     %USED     MAX AVAIL     OBJECTS 
      default.rgw.buckets     1         0         0          154G           0

  The only time the deployment worked was because the ceph-radosgw units
  were ready before gnocchi.


  
  We then tried to deploy the same bundle, but without adding the gnocchi units to the machines.

  This time ceph-radosgw did not had any issue creating its keyring and
  all pools were visible in ceph-mon

  We then added the gnocchi units to the machines, after-all the ceph charms were deployed.
  The Openstack deployment continued and worked without issue, on every tries.

  
  There maybe a race conditions in how gnocchi handle its relations.

  Environment:
  All the latest released charms on Xenial/Queen
  gnocchi-7
  ceph-radosgw-257

To manage notifications about this bug go to:
https://bugs.launchpad.net/charm-ceph-mon/+bug/1774279/+subscriptions