Juju architecture questions

Mon Sep 17 16:57:09 UTC 2012

Excerpts from Thomas Leonard's message of 2012-09-17 02:06:03 -0700:
> On 2012-09-12 20:04, Gustavo Niemeyer wrote:
> > On Wed, Sep 12, 2012 at 6:25 AM, Thomas Leonard
> > <tal at it-innovation.soton.ac.uk> wrote:
> [...]
> >> 2. I'm not sure how to create the amqp-ssl interface mentioned in
> >> https://code.launchpad.net/~tal-it-innovation/charms/precise/rabbitmq-server/ssl-support/+merge/122698
> >> only when SSL is configured.
> >
> > The interface exists at all times. The way it behaves when established
> > is a choice for the charm author. If I had to make such an SSL
> > relation available, I'd automatically build an SSL certificate on
> > first use when required, for example, rather than loading the admin
> > with further instruction on how to setup the charm.
> 
> It would be good if there were some guidance about managing keys in charms. 
> For example, should each unit have the same private key, or should they have 
> individual private keys, but all signed by the provisioning agent? Or 
> perhaps there should be a certificate chain admin->provisioning 
> agent->machine agent->unit?
> 
> If the private key is shared by all units (as currently), how do we ensure 
> that only one key is generated if multiple units are deployed?
> 

You are not the first to run up against this limitation of the 'peer'
relation type, which is meant to share data amongst units of a single
service. There is no serialization and no leader election, so you have
to assume that everybody is running in parallel.

You could fairly easily make use of simple atomic operations in any db
worth its salt to do something like this:

try:
   key = get_from_db('x')
catch FetchError:
   key = generate_key()
   try:
      add_to_db('x', key)
   catch UniqueError:
      key = get_from_db('x')

But juju won't help you do this, so you'd need to setup the db and have
it available to all service nodes for reliable operations like this.

Since Juju will always have a DB at its core (whether ZK or MongoDB) this
seems like something we should be able to do in juju fairly easily. Note:

https://bugs.launchpad.net/juju/+bug/720302

> >> What if creating the endpoint were a separate step? e.g.
> > (...)
> >> The above race condition couldn't happen, since add-relation would happen
> >> after add-endpoint had completed.
> >
> > As explained, there's no real race condition, and if we wanted to
> > modify the behavior we wouldn't really have to change the API for
> > that.
> >
> > Explicitly defining endpoints as you suggest adds yet another piece
> > onto the mental model that has to be held for comprehending the
> > system.
> 
> I'd disagree with that. For me, the most difficult part of understanding 
> Juju has been figuring out what it's doing behind the "magic". I know that a 
> database must be created; bundling it with the client connection just makes 
> it harder for me to understand what's happening.
> 

Hrm. I feel that there is a disconnect here. The idea is to automate
things, not to require administrators to specify everything. You know
that you want data to flow between your app and the database server. Do
you really care when the database is created?

> The complexity can be seen in the charm too:
> 
> # Determine if we need to create a new database
> if slave and slave_configured and not broken:
>    print "%s exists, assuming configuration done" % slave_configured_path
>    sys.exit(0)
> 
> if not slave and not broken and not admin:
>    # Find existing databases
>    cursor.execute("show databases")
>    databases = [i[0] for i in cursor.fetchall()]
>    if database_name in databases:
>      print "database exists, assuming configuration has happened already"
>      sys.exit(0)
> 

Heh, be advised that this charm is still based on what was approximately
the second charm ever written, and much of that logic came before helpers
existed to mitigate a lot of that, and also before I knew what the heck
I was doing. Most of that complexity is now nicely hidden inside juju
and those hooks need re-factoring to be dumber and just do what they're
told. :)

> # Database is created just before relation-set in case other steps fail
> ...
> 
> If endpoint creation were explicit, there would be:
> 
> - db-endpoint-added: runs exactly once, and always creates the database
> 
> - db-relation-joined: not needed?
> 

Right, but it would also be very tedious that users now have an extra
step to do what is very obvious.. you're saying to setup a relationship,
so why wouldn't the charms just *set up a relationship* ?

> > The fact we're doing well without that piece so far is an
> > indication that we can probably address the needs without making the
> > simple cases complex.
> >
> >> Adding SSL endpoint(s) to rabbit would be easy:
> >>
> >> juju deploy rabbitmq-server rabbit
> >> juju add-endpoint rabbit:ssl ssl=True key=...
> >> juju add-relation client rabbit:ssl
> >
> > It's easier today:
> >
> >      juju add-relation client rabbit:ssl
> 
> Note: today it's actually:
> 
> juju deploy rabbitmq-server rabbit
> juju set rabbit ssl_enabled=True key=...
> juju add-relation client rabbit:ssl   # (not implemented yet)
> 
> Of course, if you write the charm to make SSL available by default then it 
> doesn't make any difference.
>

What may not be obvious is that the order of the set and add-relation
need not be important. Even if you don't want to use the self-signed
cert, you can simply exit 0 on relating to the SSL relation without
setting it up fully, and wait for there to be a key, at which time the
config-changed hook can finish off the relationship with a relation-set.

> >> External clients could get the connection information even though not
> >> deployed using Juju. For example, if I want to create a database for use by
> >> a desktop client:
> >>
> >> juju add-endpoint mysql:mydb
> >> juju get mysql:mydb
> >> {JSON}
> >>
> >> Also, this would be handy:
> >>
> >> juju get nagios:admin password
> >
> > For now, you can either check the logs (which isn't nice) or set it explicitly:
> >
> >      juju set nagios admin-password=<your password>
> >
> > But there's indeed a story about pushing user-oriented details from a
> > charm to the admin that we must cover in the future. That's unrelated
> > to the other points raised in this thread, though.
> 
> I'm not sure this is true. We want to embed the nagios monitoring page in an 
> iframe from another service. I'm not quite sure how it will work, but 
> getting the connection details through the relation mechanism seems 
> sensible. In general, having to change the charms depending on whether the 
> client is or isn't managed by Juju is harder for us.
> 

The nagios charm already has an 'http' interface provider relation defined:

provides:
  website:
      interface: http

So, relate to that and it will relation-set the hostname/port that nagios
is running on.  We probably need to provide sub-paths and other bits too,
but for now, you'll at least know where to find it.

> >> Or are there already ways to do these things?
> >
> > There is for most of them.
> >
> > By the way, I'd just like to take this chance to say that I was pretty
> > impressed with your initial analysis of the system.
> >
> > Can I ask what are you planning for juju, and whether there's any way
> > we can help you moving forward?
> 
> Thanks. We make a lot of prototype systems. Typically they are made up of 
> prototype or production services developed by several different 
> organisations, which then have to be integrated. Using Juju has the 
> potential to simply this, since everyone can make their service available as 
> a Juju charm and we get a standard method for handling configuration, 
> deployment, testing, etc.
> 
> The services may or may not be open source, and we may not be able to deploy 
> them at all (we might just get an endpoint to an existing deployment). As 
> prototypes, it's usually not useful to have them in a general-purpose charm 
> store.
> 
> It's also usually uncertain where we will be deploying, so being able to 
> handle several different cloud types is useful. Often there is no cloud at 
> all, so having something which could deploy LXC containers over a fixed set 
> of existing machines would be very useful (like an extended "local" 
> deployment method).
> 

I've often thought about a more static "nonprovider" environment type
where the addresses of machines are just listed somewhere and SSH is
used to install the juju agents. It comes up often enough, I think its
time we put this on our "experiments to try soon" list.

https://bugs.launchpad.net/juju/+bug/1052065

> Our main concerns at this point are:
> 
> - security (despite being prototypes, we still need to handle "personal" 
> data in many cases),

https://bugs.launchpad.net/juju/+bugs?field.tag=security
https://bugs.launchpad.net/juju/+bugs?field.tag=production

I try to keep anything that users should read before deploying juju in
those two lists. Most, if not all, have workarounds, but its important
to read all of them before building anything significant on top of juju.