Machine id option to deploy

[Mark Ramm]

On Mon 08 Apr 2013 05:16:00 PM EST, Gustavo Niemeyer wrote:
> On Mon, Apr 8, 2013 at 7:05 PM, Mark Ramm
> <mark.ramm-christensen at canonical.com> wrote:
>> On 04/08/2013 04:19 PM, Gustavo Niemeyer wrote:
>>>
>>> On Mon, Apr 8, 2013 at 5:09 PM, roger peppe <roger.peppe at canonical.com>
>>> wrote:
>>>>
>>>> I think this is only a problem if units are ever recycled.
>>>> I think we're leaning towards the approach that a unit, once
>>>> added, lives once only and then dies forever.
>>>>
>>>> If that's the case, then the service's metadata doesn't
>>>> matter so much - it's used once to work out where to put
>>>> the unit and then is irrelevant.
>>>
>>> That's a radically different view of the metadata for an environment.
>>> Units are a by-product of a configuration.. not the configuration
>>> itself.  If there are changes going in that break that rule, you'll
>>> reach a situation where orthogonal juju goals will become harder: no
>>> more trivial add-units, no more unit-less stack configurations, no
>>> more entangled expansions of units (scale A with B), etc.
>>
>> This is a good point.  I think it is a very solid design goal of Juju to
>> make the state data the single point of truth about the system -- if it's an
>> important part of re-creating an environment it should probably be recorded
>> in the state somewhere.
>>
>> With that said, in this particular case: Once you --force-machine a unit,
>> you will definitely get some limitations, but you don't loose all of good
>> things you mention either.
>
>> You can still add-unit trivially to a service where the first machine was
>> forced to be somewhere.
>
> That's unfortunately not true, because the reason why you deployed that unit
> on that machine is lost. There are many examples of why this is the case:
>
> - If that machine died because of a hardware failure, X is gone for
> good. Juju can't
>    take any reasonable actions to recreate that unit onto a proper
> location because
>    the reason it was placed there is lost.

Agreed this is unfortunate, but it is not add-unit (which will still 
work).  That said, placement info absolutely *is* lost, so even if you 
bring it back it will be on another machine -- and I agree that this is 
far, far less than ideal.   But it is not "can't take any reasonable 
action" -- it's just "takes a less than optimal action."

I agree that "force-machine" is at best a rough hack, and a proper 
co-location story will store enough location information in state to 
make the environment re-creatable from just the info in state, on this 
point I'm completely sold.

> - If we implement the stack feature, it doesn't make sense to be talking about
>    machine X because there are no machines or unit-specific configurations.

I'm sorry, I was not here when the stack feature was first discussed, 
so I am missing a lot of context -- in Copenhagen we just got the the 
point of saying stacks were a great idea that we would not get to in 
this cycle.   My understanding of Stacks is that they would provide 
configuration information for a set of interconnected services that 
should be installed and work together, and that this would include some 
knowledge about how the whole "stack" of services scales, and perhaps 
even about how services could be co-located for efficiency.

I think it is very fair to say that proper placement/co-location is a 
prerequisite for that portion of the stacks feature, and that it would 
be incompatable with the "force-machine" style co-location contemplated 
here.


> - If the reason why it was placed in machine X is because of other
> services in there,
>    once you add-unit that service, it will fall onto another arbitrary
> location because
>    the reasoning, again, was lost.

Right, it will not be "placed" properly in the future, but it still 
will be installable in a new unit (on a new machine) using the 
constraints which were recorded. This means you can't get the kind of 
density you might want while scaling out, because that information not 
only is not recorded, but is of a type that would be completely useless 
when adding another unit (it's just a machine ID.)

> and so on.

I think we all get that this is not an ideal solution.

I wasn't here when this was discussed before, but from what I've put 
together, your argument is:

  * "force-machine" breaks juju's conceptual model
    * it does not store the reasons *why* a service was installed where 
it was
    * it introduces one time only information that *can't* be useful in 
future unit deployments
  * it results in limitations for scaling services deployed with it

  * therefore it can't be extended in a way that would allow for 
automated scaling+density

I agree with this -- it's a completely sound argument.

But here's where I see a divergence of thought: I think you are also 
arguing that because of all of the above it puts us on the wrong path, 
and therefore we should not do it.

It's at that point that I think I disagree.

As I see it "force-machine" doesn't make it harder to go down the 
proper path in the future. We can deprecate it in favour of better 
automatic placement/co-location support in the future, and provide a 
reasonable upgrade path for both Juju itself and for our users.

And offsetting it's negative side, it does solve some real problems for 
our internal Juju usage, as well as for many of our customer 
engagements.  And that feedback will help us to be able to drive Juju 
forward on this and other fronts.

Furthermore, it also gives us a better chance of moving folks from juju 
0.x to the latest Go stuff, and gets them upgrade-in-place and many 
other features that are only in our new stuff.

So, while it is not a long term solution to this problem, nor even a 
step in the right direction for solving this one problem, I think it 
puts the whole of Juju on a better path for solving real problems 
today, and ultimately will make it easier for us to get to the point 
where we are solving the co-location+isolation+density problem properly.

Hopefully that makes sense.  And I apologize if I'm missing something, 
as mentioned earlier I know this is a hot-button issue and I was not 
here for the previous round of discussion about it --  so I'm very 
aware of the possibility that I'm missing something important.  If so, 
please let me know so that I make sure I at least learn something!

--Mark Ramm