Relation ordering: Does it matter?
clint at ubuntu.com
Tue Feb 7 01:51:10 UTC 2012
So if you haven't been following the toils of Adam G in #juju, here is a
quick summary of the reason he has to control the order of relationships
in OpenStack deployment, or it won't work; This is going to bite juju
users really hard when they start to get more complicated systems that
are API driven. I don't think it has to though.
OpenStack is built around MySQL and RabbitMQ. Its API services need
these to be able to talk to all of the other services. Until the
'nova-cloud-controller' charm has a relationship fully established and
working with MySQL and RabbitMQ, it cannot answer any queries on its
API port. Likewise for keystone.
So, when you deploy nova-compute, you need to relate it to keystone. It
will then inform keystone of its existence, and get credentials back
from it or whatever it does. If you try to do that before keystone has a
database to store said information in, this will fail hard. The service
likely won't even start up without a database.
Upon establishing the database relation, there is nothing to signal to
juju that it should try again, the keystone<->compute relationship is
broken until an administrator uses 'juju resolved ...'.
This is happening because keystone is being allowed to provide services
before it has all of its required relations established.
We could just enforce that 'provides' relationships won't do anything
until a 'requires' relation exists. However, this will only narrow the
race window. Between creating the relationship, and it actually being
configured, the agents might still try to join the two together.
I think we need a way to flag a relationship as being in a steady
state. This seems quite doable given the information we have.
I've gone through this in my head a few times, and I think we can make
If all units of a relationship have exitted all pending changed hooks
with 0 changes, then we can say that the relationship is in a steady
Only upon having all required relationships in a steady state, should
the provider relations even be considered for firing their 'joined'
hooks and moving to an 'up' state.
If units are added to a required service, then the state should move
back to 'up' while any handshake occurs, and added units to services we
are providing service to must once again be delayed until the required
relations return to 'steady'.
Thoughts? If we don't do this, then we have to start defining the order
of relation establishment some other way, such as the explicit method
Adam's been using with success for openstack deployment. Even then,
there is still a race since its unknown what 'up' means, so one has to
interrogate the service nad see if it is actually ready to serve requests
to have a reliable deployment.
More information about the Juju