API change: unit and machine nodes

[William Reade]

Sorry delayed reply; nice peaceful holiday (still going, technically, but
whatever ;)).

On Mon, 2011-12-26 at 11:17 -0200, Gustavo Niemeyer wrote:
> That's not what I suggested. I'm simply advocating that the format for
> complete output be the same as for partial output, because they are
> exactly alike as far as the _data_ goes.

I'm not sure where the concept of "partial output" comes from; it seems
we have rather different perspectives on the situation. I'll try to
explain mine again:

1) We have a clear format for constraint specification at environment
and service levels. ("arch=arm"; "mem="; "ec2-zone=b"; etc).

2) Environment-level constraints are by their nature incomplete, as are
service-level constraints: they are (even together, non-exhaustive)
inputs to a formula which will eventually produce a specification that a
provider can (attempt to) convert into an actual machine.

3) Nobody has suggested there are any benefits in converting X-level
constraints into an intermediate format; after basic validation [0],
we may as well store them unchanged.

4) To produce an actual machine *specification*, we need to combine env-
and service-level constraints with the juju defaults, and we also need
to know the ubuntu series we'll be running.

5) There are several differences between level-specific constraint sets
and full machine specifications:

5a) There is no possible upside to allowing users to specify the ubuntu
series as a constraint: the series follows directly from the charm
that's being deployed.

5b) There is a user-specified constraint value that is not meaningful in
a specification (the "" (reset) value, meaning "use juju
default" (unless you feel there's a compelling reason to explicitly
expose the default values to the providers)). Similarly, missing values
are perfectly valid in a level-specific constraint set, but not in a
final machine specification [1].

5c) The actual values used to provision a machine are not necessarily
the same as those specified by the user ("12G" is a nice enough user
notation, but needs to be converted to an actual quantity of memory; "b"
is likewise merely a user-convenient shorthand for "us-east-1b" (or
possibly "sa-east-1b", or whatever)).

6) The mere *similarity* between certain features of a level-specific
constraint set and a final specification is *not* a justification for
forcing them into the same format in the name of consistency [2].

...

OK, that's the groundwork. Now, at the point where a unit is added,
*all* information required to construct a machine specification is
readily available. There is no such thing as "partial output", because
the output is unknowable until it is fully determined, and so may as
well be converted into a format more convenient for comparison and
consumption.

It seems that you can see some advantage to further deferring this
conversion, and that it's compelling enough to make it worth storing the
data -- which is IMO a very natural dict, at this point -- as a grab-bag
of datatypes which can suffer both from incompleteness and redundancy,
potentially at the same time; or which, if we complicate the code to
eliminate those flaws but preserve the original format, appears to me to
contradict your initial design advice [3] to little actual benefit.

What am I missing?

Cheers
William


[0] There's a certain amount of sanity-checking we can do at input time,
but it's somewhat limited (even if there aren't any ARM machines
available in orchestra right now, there might be come deployment time;
conversely, even if us-east-1b is available at constraint-setting time,
that's no guarantee it still will be at deployment time, but
"arch=cheese" or "ec2-zone=HAHA-DISREGARD-THAT" are fair game for
rejection).

[1] There may be disagreement here, based on the (to me) surprising
implementation of provider configuration (defaults duplicated/scattered
randomly through the provider code) but I've hitherto interpreted that
as an oversight that's not (yet) ugly/broken enough to bother fixing.

[2] Consistency's often a decent heuristic for overall sanity, but is
not infallible.

[3] You seemed to say, on launchpad [4], that even making it possible to
lossily store partial output was a problem [5]; hence the elimination of the
concept. But if we're to use the same format for constraints as for
specifications, this flaw is inescapable.

[4] https://code.launchpad.net/~fwereade/juju/constraint-types/+merge/85637/comments/185080

[5] It may be that I misunderstood that advice..?