New in 2.1-beta5: Prometheus monitoring

[Andrew Wilkins]

On Tue, Feb 7, 2017 at 6:19 PM Jacek Nykis <jacek.nykis at canonical.com>
wrote:

> On 07/02/17 02:25, Andrew Wilkins wrote:
> > Hi folks,
> >
> > In the release notes there was an innocuous line about introspection
> > endpoints added to the controller. What this really means is that you can
> > now monitor Juju controllers with Prometheus. Juju controllers export
> > metrics, including:
> >  - API requests (total number and latencies by facade/method, grouped by
> > error code)
> >  - numbers of entities (models, users, machines, ...)
> >  - mgo/txn op counts
> >
> > We're working on getting the online docs updated. In the mean time,
> please
> > refer to https://github.com/juju/docs/issues/1624 for instructions on
> how
> > to set up Prometheus to scrape Juju. It would be great to get some early
> > feedback.
>
> Hi Andrew,
>
> Thanks! Those metrics will be super useful, I will try to find some time
> to look into them properly.
>
> Some early feedback:
> 1. Your docs say the metrics endpoint requires authentication. I think
> this can be problematic for people who run multiple controllers or
> recycle them often. Secrets set up requires manual steps and they need
> to be distributed to prometheus server. It would be very useful to allow
> unauthenticated access and rely on firewalls to restrict access
> (approach followed by most prometheus exporters I looked at).
> 2. You don't offer option to downgrade to HTTP which is problematic as
> well IMO. Similar to above it's an obstacle users have to go through
> before they can scrape targets, manual steps are required, CA certs need
> to be shipped around. It would be very convenient if users could
> explicitly fall back to http and let other layers to provide security.
>
> Basically I think letting users enable unauthenticated HTTP endpoint for
> prometheus metrics would be big usability win.
>

Thanks for the feedback, Jacek.

I agree that providing unauthenticated HTTP would be helpful for many
users. I don't think that should be the default, because some of the
metrics exposed could be considered sensitive. Also, it should be fairly
straight forward to automate the configuration of the Prometheus server.

Eventually, we intend for Juju itself to be described within the model.
When that is reality, it would be sensible for the Juju controller
application to have an endpoint for unauthenticated HTTP access to metrics.
You could then just bind that to a space that Prometheus can access.

In the interim, there is https://jujucharms.com/u/axwalk/juju-introspection/.
Deploy that to any machine in Juju (including but not limited to controller
machines), and you get access to that machine agent's metrics over
unauthenticated HTTP on a configurable port. PRs welcome if it doesn't
quite fit your needs.

Cheers,
Andrew
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.ubuntu.com/archives/juju-dev/attachments/20170207/1c785a14/attachment.html>