<div dir="ltr">I'm pretty sure that is the wrong fix, as the API servers need "clusterAdmin" role so they can set up the Replica Sets when users use "juju ensure-availability". The reconfiguring is done by the machine agent, not the user admin.<div> </div><div>John</div><div>=:-></div></div><div class="gmail_extra"> <div class="gmail_quote">On Tue, Sep 9, 2014 at 2:12 PM, Ian Booth <<a href="mailto:ian.booth@canonical.com" target="_blank">ian.booth@canonical.com</a>> wrote: <blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">Before I sent the original email I put up a PR to reduce the noise by eliminating the warnings. eg we only apply the cluster admin role to the admin database, not the presence or juju databases. <a href="https://github.com/juju/juju/pull/714" target="_blank">https://github.com/juju/juju/pull/714</a> This cuts down on the warnings but we still get repetitive syslog entries for mongo authentication on the connections. I also put in logging to see if we were calling login over and over and it appears we are not. The agent is not bouncing. From what I can tell, Juju only calls Login a finite number of times when the agent starts. Everything is working as expected. We do call session.Copy() when we perform a db interaction, but that to my understanding just reuses the credentials from the original session. Maybe that also causes mongo to create these log entries? >From what I can see, this is a mongo logging issue that we have little control over. But I'd love to be wrong so we can fix it. <div><div class="h5"> On 09/09/14 18:53, John Meinel wrote: > On Tue, Sep 9, 2014 at 12:27 PM, Andrew Wilkins < > <a href="mailto:andrew.wilkins@canonical.com">andrew.wilkins@canonical.com</a>> wrote: > >> On Tue, Sep 9, 2014 at 4:14 PM, John Meinel <<a href="mailto:john@arbash-meinel.com">john@arbash-meinel.com</a>> >> wrote: >> >>> I don't think these are from authentication between mongo instances. >>> >> >> Agreed, looks like the machine-0 jujud connecting over and over again. >> Perhaps the agent is unhappy and is bouncing. >> > > It is possible, but its also possible that the new session copying means we > just have multiple connections (conn11) to the database, and each one is > going to be authenticated. > > >> >> >>> This looks more like a Mongo 2.4 vs 2.6 difference. We added stuff like >>> "clusterAdmin" to priviledges that we give because of 2.6 support (I >>> thought).. But if 2.4 doesn't know anything about "clusterAdmin" then >>> whenever we log in, it goes through the list of roles and noisily ignores >>> some of them. >>> >> >> clusterAdmin is for replica sets, and is applicable to 2.4. In particular, >> in grants replSetInitiate. I went through and removed the extraneous stuff >> when I did the 2.6 support updates. >> See <a href="http://docs.mongodb.org/v2.4/reference/user-privileges/" target="_blank">http://docs.mongodb.org/v2.4/reference/user-privileges/</a> >> > > Digging into it a bit more, it is only complaining about not having those > roles on the "juju" and "presence" databases, they are probably present on > the "admin" database. > > Looking at the code, the only place I see RoleClusterAdmin is while > connected to the Admin database. > "clusterAdmin </div></div>> <<a href="http://docs.mongodb.org/manual/reference/built-in-roles/#clusterAdmin" target="_blank">http://docs.mongodb.org/manual/reference/built-in-roles/#clusterAdmin</a>> is <div class="HOEnZb"><div class="h5">> only applicable on the admin database, and does not confer any access to > the local or config databases." > > I wonder if this is either: > 1) Mongo just being overly verbose in warnings that aren't actually > warnings. (You logged in with a user that has clusterAdmin, but that > doesn't apply to this other database you are accessing.) > 2) We somehow created a User on the Juju/Presence databases and then also > gave it "clusterAdmin" rights there. > > The only place I see any of this is via the mongo.SetAdminMongoPassword > call, though, and that does appear to forcibly only do the UpsertUser > rights on "admin". > So I'm *guessing* we are running into (1). We are connecting directly to > the "juju" database, and we are connecting as a user that has clusterAdmin > but it just doesn't do anything in this context and mongo warns about it. > An idea is that we always connect to "admin" and then switch databases once > connected? I think that's how I've done it for connecting when using > "mongo" the CLI client. > > John > =:-> > > > > >> Presumably the real fix is to only give roles that are 2.6 specific when >>> we know we're running against 2.6 ? >>> >>> (I could be wrong on this, but I thought we had earlier bugs that we >>> couldn't do the cluster reconfiguration on 2.6 because we had to add the >>> "clusterAdmin" role.) >>> >>> John >>> =:-> >>> >>> >>> On Tue, Sep 9, 2014 at 11:22 AM, Ian Booth <<a href="mailto:ian.booth@canonical.com">ian.booth@canonical.com</a>> >>> wrote: >>> >>>> <a href="https://bugs.launchpad.net/juju-core/+bug/1365828" target="_blank">https://bugs.launchpad.net/juju-core/+bug/1365828</a> >>>> >>>> Hi mongo experts >>>> >>>> Mongo is filling up syslog with lots of spam. >>>> eg, as reported by a customer, <a href="https://clbin.com/XZ63i" target="_blank">https://clbin.com/XZ63i</a> >>>> >>>> I have reduced this a little by tweaking the roles used to log in to the >>>> "juju" >>>> and "presence" databases, so the warnings are gone. But the spam remains. >>>> >>>> It seems to me these Mongo bugs might be relevant: >>>> >>>> <a href="https://jira.mongodb.org/browse/SERVER-5847" target="_blank">https://jira.mongodb.org/browse/SERVER-5847</a> >>>> <a href="https://jira.mongodb.org/browse/SERVER-5952" target="_blank">https://jira.mongodb.org/browse/SERVER-5952</a> >>>> >>>> I can't see from a brief look what Juju might be doing to directly cause >>>> the >>>> spam, nor what we can do to remove it. >>>> >>>> Does anyone with more Mongo knowledge have any insight as to what we can >>>> do to >>>> address this problem? >>>> >>>> >>>> -- >>>> Juju-dev mailing list >>>> <a href="mailto:Juju-dev@lists.ubuntu.com">Juju-dev@lists.ubuntu.com</a> >>>> Modify settings or unsubscribe at: >>>> <a href="https://lists.ubuntu.com/mailman/listinfo/juju-dev" target="_blank">https://lists.ubuntu.com/mailman/listinfo/juju-dev</a> >>>> >>> >>> >>> -- >>> Juju-dev mailing list >>> <a href="mailto:Juju-dev@lists.ubuntu.com">Juju-dev@lists.ubuntu.com</a> >>> Modify settings or unsubscribe at: >>> <a href="https://lists.ubuntu.com/mailman/listinfo/juju-dev" target="_blank">https://lists.ubuntu.com/mailman/listinfo/juju-dev</a> >>> >>> >> > </div></div></blockquote></div> </div>