[Canonical-juju-qa] Cursed (final): #1479 gitbranch:master:github.com/juju/juju ead2e2d6 (functional-ha-recovery, hp-upgrade-precise-amd64)

[Ian Booth]

> ...
> 2014-06-14 10:06:11 ERROR juju.worker runner.go:218 exited "peergrouper":
> cannot get replica set status: cannot get replica set status: unauthorized
> 2014-06-14 10:06:11 INFO juju.worker runner.go:252 restarting "peergrouper"
> in 3s
> 
> IIRC Ian did some changes wrt how we handle users in an attempt to make our
> code work with Mongo 2.6 (that changed UpsertUser vs AddUser vs whatever
> else we were using). I think it is supposed to work in other versions of
> Mongo, but I'm guessing it isn't quite as compatible as it is supposed to
> be.
> 

Thanks for digging into this John. Yes, it was due to the change I made to
migrate us to using UpsertUser from AddUser :-( AddUser is deprecated in mongo
2.4 in favour of UpsertUser (according to the mgo driver). UpsertUser requires
explicit specification of user roles. The roles I assigned  were:
mgo.RoleReadWriteAny, mgo.RoleUserAdmin

It seems I also needed to include mgo.ClusterAdmin. I did have this role
assigned to the admin user which we create using our SetAdminMongoPassword API
but didn't realise I needed to assign it to a non-admin user.

When I was setting up the roles, I did get various unit test failures and once
they passed I assumed that everything was then correct. We do have replicaset
tests in our test suite so I guess I assumed that these would have failed if
there were any issues with user setup. In my fix, I've augmented the user
creation tests with checks for assigned roles.

One thing that I've noticed looking into this is that with the change from
AddUser to UpsertUser, we now have the chance to tighten up our granting of
permissions. We have this api call in Juju:

SetMongoPassword(name, password string, dbs ...*mgo.Database)

It appears this is what's used to universally set up the credentials for all of
our agents: unit, machine etc. With AddUser, we could only say if the user was
readOnly or not. And we used readOnly=false so all users we created must have
had all permissions. Which means that we are giving permissions to users which
don't need them. eg why would a unit agent need to be able to manage the
cluster. So I think there's room for improvement here and we can use UpsertUser
to do it.