[Bug 1786146] Re: ceilometer meter_time_to_live has no effect if applied to existing mongodb database

[Launchpad Bug Tracker]

Status changed to 'Confirmed' because the bug affects multiple users.

** Changed in: ceilometer (Ubuntu)
       Status: New => Confirmed

-- 
You received this bug notification because you are a member of Ubuntu
OpenStack, which is subscribed to ceilometer in Ubuntu.
https://bugs.launchpad.net/bugs/1786146

Title:
  ceilometer meter_time_to_live has no effect if applied to existing
  mongodb database

Status in ceilometer package in Ubuntu:
  Confirmed

Bug description:
  If metering_time_to_live is left at its default (never expire data)
  and later switched to a positive value, it will have no effect.
  ceilometer implements expiry by using a mongodb TTL index.  However,
  it does so when there is already an existing index on the timestamp
  field: https://pastebin.canonical.com/p/4vJNXQgDNM/

  https://docs.mongodb.com/manual/core/index-ttl/ states "If a non-TTL
  single-field index already exists for a field, you cannot create a TTL
  index on the same field ... To change a non-TTL single-field index to
  a TTL index, you must drop the index first and recreate with the
  expireAfterSeconds option."

  This index was added in the fix for lp:1193906; however those commits
  did so without taking into account the TTL index.

  In the default configuration of ceilometer and ceilometer-charm, this
  isn't a problem, because data is never expired.  But because the
  manual expiry of data is gated on whether TTL indexes are supported by
  the mongodb version in use (see _is_natively_ttl_supported in
  https://github.com/openstack/ceilometer/blob/e984f28b00e3a073c176f6435507f5bce1c82d30/ceilometer/storage/impl_mongodb.py),
  positive TTLs can result in data not being expired correctly.

  Ideally, the code should be updated to create a TTL index if it is
  supported and a positive TTL is configured, and a non-TTL index if it
  is not supported (mongodb versions before 2.2), or TTL is non-
  positive.

  Because mongodb is a deprecated storage driver for ceilometer, this
  isn't likely to be high priority or to be accepted by upstream, so
  I'll document the workaround here:

  - Check if there's a TTL index on mongodb:

  myset:PRIMARY> use ceilometer
  switched to db ceilometer
  myset:PRIMARY> db.meter.getIndexes()
  ...

  - If an index called "meter_ttl" is present, ceilometer has configured
  a TTL index.  Drop the non-TTL index:

  myset:PRIMARY> db.meter.dropIndex("timestamp_idx")

  Unfortunately, there's no indication in the mongodb documentation as
  to what will happen if "timestamp_idx" is recreated (which ceilometer
  seems to do on connection to mongodb).  Hopefully the presence of the
  "meter_ttl" index should prevent "timestamp_idx" from breaking an
  existing TTL index, but I don't have any evidence to suggest one way
  or the other yet.

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/ceilometer/+bug/1786146/+subscriptions