Supported container types for images

[Scott Moser]

On Thu, 30 May 2013, Ian Booth wrote:

> Hi folks,
>
> You will be aware we are implementing containers for Juju. Machines can be asked
> to create an LXC, KVM, XEN (et al) container. Not all machine instances will be
> able to run all containers. eg kernel patches may be required to support XEN or
> whatever and the image may not have been set up with that capability. Or LXC
> doesn't make sense on Windows images etc.
>
> Juju needs to know what types of container an instance can run. Ideally, we
> would know this before starting an instance from an image, since the add-machine
> command would optionally be asked to create a machine with a given container.
> We'd love to be able to get this information from the simplestreams image
> metadata. Theoretically it should be possible because whoever creates the images
> knows their capabilities. And we already have access to this kind of data now,
> like the storage type or virtualisation type (for EC2). The alternative is to
> wait for a machine instance to boot and query the capabilities then but this
> opens up a whole world of hurt we'd rather not deal with.
>
> So, is what we're asking for viable? Can the simplestreams image metadata be
> made to include the types of supported containers? Is there a better way to
> achieve our goals?

First the little thing.  At the moment, 'items' in simplestreams's
'products:1.0' format have key/value pairs associated with them.  At the
moment all those values are strings, but your "list of supported
virtualization types" would be best modeled with a key:value where the
value was a list rather than a string.

Now, the larger thing.

As a simplifying assumption, lets use the definition of "image" that is
understood from "ami" (amazon machine image), which is not terribly
different than the 'image-id' that openstack has.  Basically:
 1. an image-id is really a block device mapping, where that is one or more
    collections of immutable bits to be attached as block devices on
    instance boot.
    [snap-abcdefg: /dev/sda, s3://foo.bar/manifest.xml: /dev/sdb]
 2. an image-id can be booted inside one or more instance-types, an
    instance type being 'm1.small', or 'm1.large', or 'cc4.4xlarge'.

An instance-type is really quite similar to a "provided hardware
platform".  On Amazon, at the moment, it represents things like:
  m1.small: xen para-virt, x86_64, 2G ram
  m1.large: xen para-virt, x86_64, 8G ram
  cc4.8xlarge: xen hvm-virt, x86_64, 8G ram

It could also represent things like:
  k1.large: kvm with-nested-kvm, x86_64 2G
  h1.large: x86_64 hardware 16G
  xx2.large: xen dom0, x86_64, 16G

A given 'ami' could actually be made to boot actually boot on any of
those, and that is not terribly outside my goal for the ubuntu images.
In a perfect world, we have 1 cloud image that "just works".

Right now, simplestreams contains information about an AMI.  It does not
contain any information about instance types.  Thats been requested, we
dont have it right now.

What I'm getting at, is that specifically 'xen' or 'kvm' support are
dependent upon the "provided hardware platform"/instance-type that the ami
boots on.  For xen container support, you'd have to boot on one of
'h1.large' or 'xx2.large' above.  And for kvm container support, you'd
need 'k1.large' or 'xx2.large'.

lxc containerization is a different beast in that it is pretty much
"provided hardware platform" agnostic.  IMO it is probably sufficient at
this point for juju to "just know" that
'com.ubuntu.cloud:server:12.04:i386' has lxc support (at a given lxc
version).  A given 'product_name' like that is expected to behave in a
consistent way wherever you see it.

I'm not entirely opposed to putting some attributes of the installed
software into simplestreams data, but I think its potentially more
complexity than necessary.  At least at the moment.


Now, for a solution, I think we have to have machine formated
data describing a instance-type for each cloud (or endpoint/region pair).
Then, juju could find an ami with product_id of
'com.ubuntu.cloud:server:12.04:amd64' and know that it could run that on a
h1.large and be able to create containers using kvm 1.1.
and know that if it ran it on a h1.large