[Bug 1946793] Re: aodh uses deprecated gnocchi api to aggregate metrics and doesn't work properly

[Gustavo Sanchez]

Sorry, forgot to add aodh logs

** Attachment added: "logs.tar.xz"
   https://bugs.launchpad.net/aodh/+bug/1946793/+attachment/5560923/+files/logs.tar.xz

-- 
You received this bug notification because you are a member of Ubuntu
OpenStack, which is subscribed to aodh in Ubuntu.
https://bugs.launchpad.net/bugs/1946793

Title:
  aodh uses deprecated gnocchi api to aggregate metrics and doesn't work
  properly

Status in Aodh:
  New
Status in aodh package in Ubuntu:
  Incomplete

Bug description:
  On gnocchi API docs, there are 2 API methods to aggregate metrics

  1. /v1/aggregation/metric?

  See: https://gnocchi.osci.io/rest.html#aggregation-across-metrics-
  deprecated

  This one is deprecated

  2. /v1/aggregates?

  See: https://gnocchi.osci.io/rest.html#dynamic-aggregates

  aodh uses the 1st one to aggregate metrics, for example:

  ```
          if isinstance(start, datetime.datetime):
              start = start.isoformat()
          if isinstance(stop, datetime.datetime):
              stop = stop.isoformat()

          params = dict(start=start, stop=stop, aggregation=aggregation,
                        reaggregation=reaggregation, granularity=granularity,
                        needed_overlap=needed_overlap, groupby=groupby,
                        refresh=refresh, resample=resample, fill=fill)
          if query is None:
              for metric in metrics:
                  self._ensure_metric_is_uuid(metric)
              params['metric'] = metrics
              measures = self._get("v1/aggregation/metric",
                                   params=params).json()
  ```

  aodh doesn't work properly in our production environment after
  upgraded to Ussuri.

  When there is only 1 instance, aodh works properly and alarms can be
  triggered when the load on the instance is higher than the threshold.

  However, after the stack is scaled up, and the second instance is
  created. The average cpu usage got from gnocchi by aodh evaluator is
  not correct. The metric measures are negative sometimes.

  I manually pulled metrics with gnocchi command

  The aggregation of metrics is correct with command

  ```
  openstack metric aggregates
  ```

  It uses new API in the backend

  The aggregation of metrics is not correct with command

  ```
  openstack metric measures aggregation
  ```

  It uses the deprecated API which aodh is using.

To manage notifications about this bug go to:
https://bugs.launchpad.net/aodh/+bug/1946793/+subscriptions