machine placement spec
william.reade at canonical.com
Fri Nov 11 14:56:35 UTC 2011
On Wed, 2011-11-09 at 17:24 +0100, William Reade wrote:
> Hi all
> Here's a first attempt at a spec for machine placement. I'd be very
> grateful for your comments, and opinions on which (if any) of the
> described enhancements is valuable enough to be promoted to the main
> part of the spec.
> It's available at docs/source/drafts/placement.rst in
> lp:~fwereade/juju/placement-spec; pasted below for convenience.
...and here's the latest version, incorporating the recent discussions
(I think; let me know if I've missed anything).
To fulfil a popular set of use cases, juju needs a mechanism to control
placement of service units; to fulfil the simplest possible version of
this story, we consider two possible cases:
* User needs to specify minimum hardware requirements
* User needs to specify other, context-dependent, hardware attributes
This feature allows us to remove the temporary EC2 environment
properties `default-image-id` and `default-machine-type`, and makes
obsolete the requirement to interact with cobbler out-of-band when
placing services and units in an orchestra environment.
We introduce "machine constraints", which are used to encode an
administrator's hardware requirements. Constraints can be set for
environments and services, with lookups for each key falling back from
more specific to more general settings, with default values set by juju
when otherwise unspecified _. Changes to constraints do not affect
any unit that's already been placed on a machine.
Environment constraints, like other environment settings that are not
related to access, will be controlled with a new command, `juju env-set`
_; the existing `juju set` command will be used for service
constraints. Any number of constraints can be set in a single
`--constraints` option, as a space-separate list of `key=value` pairs.
An empty `value` is treated as "the juju default", allowing users to
ignore environment settings at the service level without having to
explicitly remember and use the juju default value.
We also extend the syntax for `juju deploy`, such that `--constraints`
is understood as above; these constraints will be set on the service
before the first unit is deployed.
The constraints fall into two categories:
* Generic constraints: based on the minimal properties exposed by every
provider. These are:
* `cpu`: The minimum processing power of the machine, measured in `ECU
http://en.wikipedia.org/wiki/Amazon_Elastic_Compute_Cloud#Elastic_compute_units`_, defaulting to 1; any real number >= 0 is valid.
* `ram`: The minimum memory for the machine, defaulting to 512MB; any
real number >= 0 and suffixed with M, G or T is valid.
In each case above, the value `0` is treated specially, and used to
denote that the value doesn't matter at all. EC2 could, for example,
special-case `cpu=0` to allow for deploying to a t1.micro without naming
it explicitly, but this is not strictly necessary; in the general case,
`<key>=0` simply means "do not filter on `<key>`.
When interpreting generic constraints, juju is expected to deploy
machines with as little wastage as possible: that is to say, given a
`ram=3G` constraint and two available machines with 4G and 6G
respectively, juju would deploy to the 4G machine.
* Provider constraints: based on the additional properties exposed by a
given provider, which are not expected to be directly portable to other
* For EC2, we expect to expose:
* `ec2-region`: the region in which to deploy new instances.
Defaults to "us-east-1"; valid values are the current set of suported
* `ec2-zone`: availability zone within the region. Defaults to "a";
valid values depend on the current value of `ec2-region`.
* `ec2-instance-type`: instance type. Unset by default (so we fall
back to the juju-level `ram` and `cpu` requirements); valid values are
those machine types that EC2 has available in the given region.
* For orchestra, we expect to expose:
* `orchestra-name`: to allow admins to specify a specific instance
by name, if required. Unset by default; the valid values are the set of
instance names exposed by cobbler.
* `orchestra-classes`: a comma-separated list of cobbler
mgmt-classes to which the instance must belong. Empty by default; the
valid values are the existing cobbler mgmt-classes, excluding the
current values of `available-mgmt-class` and `acquired-mgmt-class` (from
the orchestra environment settings).
Provider constraints are only effective when used with the appropriate
provider, and are silently ignored when specified with a different
Note that provider constraints can overlap with generic constraints:
it would not be possible to deploy a machine with the constraints
`ram=8G` and `ec2-instance-type=m1.small`, because the requirements are
Finally, please note that for the local LXC provider, given its nature
and its intended use as a development tool, no constraints have any
Please note that this spec depends on features of juju and orchestra
that are not present at the time of writing:
* We need orchestra to expose `cpu` and `ram`; as long as the numbers
are available, the orchestra team is free to expose them in whatever way
is convenient; but the feature will be severaly hobbled if we don't have
access at all.
* Environment settings will need to be rearranged, and our handling of
`environments.yaml` will need to change, so that we can separate an
environment's access information from the runtime properties contolled
by `juju env-set`.
The above spec is intended to be small and focused; several enhancements
are possible and may be desirable in the future. They include:
* Unit-level constraints, specified at `juju add-unit` time, and
inheriting naturally from service, environment, and juju.
* An additional generic constraint, `storage`: The minimum persistent
disk space for the machine, defaulting to 4GB. Valid inputs as for
* Max constraints: allow generic constraints to also take the value
`max`, meaning "the best available". (If you specify both `cpu=max` and
`ram=max`, the constraints cannot be satisfied unless there is an
available machine with the (equal) greatest amount of RAM and also the
(equal) most processing power; in general, we would expect people to
only set one constraint to `max`.)
* Additional provider constraints, such as:
* `orchestra-dns-name`: potentially useful for sysadmins who don't
primarily think of their systems by `orchestra-name`.
* Co-location constraints: for determining machine placement in terms of
existing juju components. Possibilities include:
* `place-in=<machine-id>`: On a separate container in the machine with
juju id `machine-id`.
* `place-with=<service-name>`: On a separate container in *any*
machine with a unit of `service-name` deployed.
* `scale-with=<service-name>`: On a separate container in *every*
machine running a unit of `service-name`; henceforth every add or remove
of a unit of `service-name` will lead to the addition or removal of the
corresponding unit of the requested service.
Note that it is not advisable to implement co-location constraints
without having the ability to deploy units in isolated containers on
single machines; doing this is somewhat tricky.
* Provider->generic constraint translation: In the event of our
implementing stacks, it may be useful to be able to convert
provider-specific constraints into generic ones (where possible) to
facilitate creation of provider-independent stacks.
* Named groups of constraints, provisionally named "roles": Useful for
OAOO-ness and reduced typing when scripting or running from the command
line; also useful for recording intended machine characteristics when
not otherwise translatable (for example, when serialising a deployment
as a stack, it would be thoughtful to include a `fast-network` role to
hold the `orchestra-classes=rack-c` constraint which *you* know means
"next to the switch" but is meaningless to people outside your
..  So: if an environment has specified `ec2-zone=a ram=1G` and a
service has specified `ram=2G`, instances of that service in that
environment will inherit the `ec2-zone` setting of "a", and the juju
default `storage` of "4G", while using the locally-specified `ram` of
..  `juju env-set` is intended to take on reponsibility for a number
settings previously only exposed via `environments.yaml`; these
environment settings will henceforth be exposed via `juju set-env`, and
can also be passed into `juju bootstrap`, using the same syntax.
More information about the Juju