set_state not setting the state immediately

fengxia fxia1 at
Tue Oct 31 13:39:06 UTC 2017


Nice summary of these issues. Especially for the #2, we have been using 
Ansible (called from charm) for executions. I think there are 3 points 

1. confg mgt has solved idempotency, great!
2. many organizations have accumulated an asset of recipes using these 
config tools (puppet, chef, ansible, and so on)s. Being able to re-use 
them makes transition and adoption within these orgs easier.
3. Betting on technology. Puppet, chef, ansible all have/had their days. 
New things will inevitably emerge. Having two in the stack, though 
inefficient, mitigate risk. This is just sad reality.

Using an analogy. A charm is like a cooked dish prepared by a wonderful 
chef, tastes great, but few can do that at home yet. I have awed many by 
showing them how easy it is to deploy an Openstack with a click of 
button using Juju GUI. But ppl fear things they don't understand. Config 
mgt tools are like giving you a set of cooking utensils and a recipe 
book, you can follow the book, or be an iron chef yourself, and 
surprisingly, many believe they have that talent to be the iron chef, 
though over optimistically often ~~

On 10/30/2017 11:56 AM, Merlijn Sebrechts wrote:
> Hi all
> Great discussion in this thread. I sense there are two issues here:
> 1. The transactional nature of charms.reactive and Juju needs to be 
> explained better 
> <>. We 
> can't change the transactional nature from the charms.reactive side 
> since this is a Juju core feature, but we can provide a lot better 
> docs and change function names to better match their actual behavior. 
> This is already discussed for relationships as part of the Endpoint PR 
> <>.
> 2. Idempotency is hard, not commonly understood outside of config mgmt 
> community and charms.reactive isn't helping. 
> <> Idempotency 
> is a solved issue in config mgmt tools. I don't think it's the job of 
> Juju and charms.reactive to provide ways to do this because we operate 
> on a higher level (service orchestration, not config mgmt). What we 
> should to is make it easier to use charms.reactive together with 
> config mgmt tools like Puppet and Chef. This will keep us from 
> reinventing the wheel and will provide a number of additional benefits 
> (such as being able to leverage existing Puppet scripts and quicker 
> charms).
> Please create more issues if I've missed something, and add your 
> comments to the issues.
> Kind regards
> Merlijn
> 2017-10-05 20:50 GMT+02:00 fengxia <fxia1 at 
> <mailto:fxia1 at>>:
>     " An assumption is being made that the state changes get committed
>     immediately, but these changes are actually transactional and
>     following the same transactional behaviour as the Juju hook
>     environment [1]."
>     To chip in my experience as 6-month into learning charms and
>     writing a few simple charms.
>     1. The "transactional" nature of Juju hook needs better explained.
>     To be honest I have no idea what this means, and what implication
>     it has to a charm writer. Any reference would be helpful.
>     2. I like Mike Wilson's approach to provide a list of
>     "set_state_xxx" functions so new writer can better guess what this
>     function will do. Further, a different name calls for further
>     study why they are different, thus learning important concept of
>     whatever Juju thinks charm writer needs to understand.
>     Otherwise, I will expect "set_state" will set that state/flag
>     asap. If there is a scanning cycle (which I heard there is some
>     kind of 5-min cycle, which document has not sufficiently made it
>     clear to a writer either), charm writer needs to have better doc
>     to learn what it means for design. I came from embedded system
>     world in which a timer loop is common. It calls for a different
>     thinking than user space script user. I think such implication
>     should be emphasized more.
>     3. idempotent
>     Again, this is a concept me (or many new writer) will fail to
>     grasp. Looking at "apt install" as example, my reaction was that
>     the package manager is taking care of "doing nothing" if called
>     multiple times. But how this translate to my system in which charm
>     is expected to "do something"? Does it mean I need a gatekeeper
>     like the package manager so to guard these "multiple calls"?
>     Again, this feels like a work around because "set_state" will call
>     the same function block multiple times, which is unintuitive to
>     writer -- when I set a state, the action for that state is
>     executed once, not over and over again until I turn it off.
>     Further, even "remove_state" doesn't take effect immediately, so
>     it feels arbitrary how many cycles a block of code is executed.
>     This is a design pattern I'm afraid many are not familiar with, so
>     some tutorial examples will be much appreciated.
>     Best,
>     Feng
>     On 10/04/2017 08:59 AM, Marco Ceppi wrote:
>>     So, I've not actually checked the logs in a while, but if
>>     visibility is an issue, it seems reasonable that the reactive
>>     framework should print in the log that X flags are being reset
>>     due to hook failure. Things like set_flag_immediate has farther
>>     reaching consequences than simply stating that flags only get set
>>     on success of a hook run.
>>     I know there are further reaching initiatives to alleviate this
>>     by decoupling the reactive state engine from the juju hooks
>>     system. In this case each successive loop of the reactive runtime
>>     could better snapshot state and make failures more granular.
>>     * state is being renamed to flag in the next major version of
>>     reactive.
>>     On Wed, Oct 4, 2017 at 8:52 AM Mike Wilson
>>     <mike.wilson at <mailto:mike.wilson at>> wrote:
>>         So as a new charm writer coming to Juju I would first do this:
>>         def get_ready():
>>         func0()
>>         func1_fails()
>>         Then I would, hopefully, test and notice issues. I would
>>         investigate and see that I needed to be idempotent. My next
>>         attempt would be to wrap those functions inside state checks
>>         with sets after they complete. This would also fail and now
>>         the charm creator is left with nothing in the api that can
>>         help. They are now off to their own devices to start doing
>>         random things to attempt to make this work the way they want
>>         it to work. Hopefully, the solution is as straight-forward as
>>         touching random files, but we just never know.
>>         I would expect the name of set_state to be something like
>>         set_state_on_success and I would further expect some sort of
>>         immediate state thing like set_state or set_state_immediate.
>>         This would give the user the tools we know that they need in
>>         order to create bug-free charms.
>>         Now to compound that confusion, we have the issue of a hook
>>         can call multiple functions inside the charm code and if any
>>         of those functions have something that fails the whole thing
>>         is unwrapped. I understand from a Juju perspective why this
>>         is the case, but as a user, I would be completely taken by
>>         surprise here. The only real fix here is documentation so
>>         that we can set expectations, but people will most likely
>>         look at examples instead of documentation. This means that we
>>         need to make sure to call out any potential issues like this
>>         in the example charms we release.
>>         On Wed, Oct 4, 2017 at 6:34 AM Stuart Bishop
>>         <stuart.bishop at
>>         <mailto:stuart.bishop at>> wrote:
>>             On 4 October 2017 at 00:51, Mike Wilson
>>             <mike.wilson at
>>             <mailto:mike.wilson at>> wrote:
>>             > So the best practice here is to touch a file and test
>>             for the existence of
>>             > that file before running must_be_called_exactly_once()?
>>             >
>>             > I think part of the issue here is that without knowing
>>             the extent of the
>>             > hook it is hard to enforce idempotency as a charm
>>             writer. It's easy to look
>>             > at the code above and say that is it idempotent since
>>             the init function is
>>             > wrapped in a when_not and the initialized state is set
>>             at the bottom of
>>             > init.
>>             Individual handlers should be idempotent, so it doesn't
>>             matter about
>>             the extent of the hook, or even if the chained handlers
>>             being triggers
>>             are running in the same hook. Assume your handlers get
>>             called multiple
>>             times, because they may be. Yes, it looks idempotent but
>>             it isn't. An
>>             assumption is being made that the state changes get committed
>>             immediately, but these changes are actually transactional and
>>             following the same transactional behaviour as the Juju hook
>>             environment [1]. I think this can certainly be explained
>>             better in the
>>             docs, but I can't think of a way to stop this being an
>>             easy error to
>>             make.
>>             [1] spot the DBA
>>             --
>>             Stuart Bishop <stuart.bishop at
>>             <mailto:stuart.bishop at>>
>>         --
>>         Juju mailing list
>>         Juju at <mailto:Juju at>
>>         Modify settings or unsubscribe at:
>>         <>
>     -- 
>     Feng Xia
>     Advisory Engineer
>     Datacenter Group (DCG), Lenovo US
>     8000 Development Dr, Morrisiville, NC 27509
>     W:
>     --
>     Juju mailing list
>     Juju at <mailto:Juju at>
>     Modify settings or unsubscribe at:
>     <>

Feng Xia

Advisory Engineer
Datacenter Group (DCG), Lenovo US
8000 Development Dr, Morrisiville, NC 27509

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <>

More information about the Juju mailing list