Splitting cloud credentials out of config.Config

[Tim Penhey]

Hi All,

This is a bit of a brain dump and thoughts around this.  Some of this
may be a bit rambly...

The cloud credentials are not needed by any worker other than the
provisioner and firewaller (I think).  If they are, the worker should be
one that is only run on the main state server machines.  It is not
desirable to share these credentials with other parts of the system
where they are not needed. In particular keeping the credentials away
from anything that deploys charms (yes I know that it is possible to
deploy charms on state server machines, but it doesn't make it good
secure practice).

These credentials are normally marked as "SecretAttrs" which are
stripped from the environment's config for the watchers.

If we had account credentials as a separate map, stored separately in
the DB, and only accessible to those workers that truly needed them,
then we'd be in a more secure position.

We also want to move towards having accounts defined in their own
configuration file.  This would allow the creation of environments with
default args in a more simple manner.

I don't think that the idea of separating out the cloud credentials
information from the general environment config is a controversial idea,
but we need some thought about how to get there from here.

It is simple enough to define a collection that will hold account
information for an environment.  Perhaps even we store the account
information with the environment itself, although if we want watchers on
it, it probably makes sense to have a separate collection.  We should
also consider the future where there may be multiple accounts associated
with a given environment.  It is much simpler to plan for multiple and
use one than to plan for one and have to fix it later.

An upgrade job could relatively simply remove the attributes from the
environment config and make them available through this account
(credentials) config.  Since all the upgrade jobs happen before the
other workers are created, we would be able to ensure that the workers
in the new version are constructed with the account information they need.

What is needed is some intermediate step that can take the current
environments.yaml config values and split that into the environment
config and account credentials.  This would need to be provider specific.

The providers, when created would need to be created with both
environment config and account credentials.  This way, when juju moves
to the new world of account configuration files, the provider factory
methods don't need to change.

I'd like to get a sanity check on this before I start messing around
with the code.

Thoughts?

Tim