upcoming changes to Ubuntu cloud images via cloud-init

[Scott Moser]

[Replying and adding ubuntu-cloud and ubuntu-cloud-announce]

In an effort to improve the performance and efficiency of Ubuntu's cloud
images[1], cloud-init is making some changes in the way that it searches
for datasources.  By making use of environment information that platforms
provide such as DMI (Desktop Management Interface) data, cloud-init can
quickly filter out datasources that are not present.

The driver for doing this is to:

 - Improve boot speed of Ubuntu images by quickly discarding datasources
   that are not present.
 - Avoid attempting to access network resources that may not be present.
   That can result in timeouts or other unexpected behavior.
 - Entirely disable cloud-init if no datasource is present.

Examples of such information that cloud-init will consider:

 - OpenStack Nova populating the DMI system product name 'OpenStack Nova'.
 - Google Compute Engine [2] populating the DMI system product name with
   'Google Compute Engine'.
 - Azure guests will be running under Hyper-V and contain a disk with a
   well known filesystem label.
 - Amazon EC2 [3] populating the DMI system uuid with the same value as
   system serial number, and will start with 'ec2'
 - NoCloud datasource has disk attached with a well-known label 'cidata'.

Unless those datasource identifiers are present, the datasource will not
be queried.

As with all things in Ubuntu, has been made first in the development
release (17.04). Eventually it will make its way into the 16.04 LTS
release by way of a Stable Release Update.

In the past some cloud providers have served metadata to instances running
unmodified Ubuntu images by emulating the EC2 metadata service.  With the
plan above in place, those instances will no longer receive metadata
without modification to either the image or the environment.

Our goal is to work with all cloud providers to ensure that their
environments enjoy the performance improvements these changes to Ubuntu
and cloud-init provide.  The best practice for cloud providers is to
supply a specific string in DMI data to their guests that will allow
cloud-init to identify which datasource should be used.

To make sure that instances booting from Ubuntu cloud images continue to
work on your cloud, please:

Test the Ubuntu 17.04 (Zesty) cloud images from
https://cloud-images.ubuntu.com/zesty
If you experience problems, open a bug [4] against cloud-init and provide
information on what datasource cloud-init previously used as well as how
cloud-init can identify your platform.

After you've filed a bug with the proper information, you can make custom
images which will continue to work by specifying the datasource that is
expected to be used.  To do so, simply write the following to
/etc/cloud/cloud.cfg.d/99-datasource.cfg, replacing 'YourCloud' with a
datasource below:

datasources_list: [YourCloud, None]

The accepted datasources in 17.04 are:
  AltCloud, Azure, Bigstep, CloudSigma, CloudStack, ConfigDrive,
  DigitalOcean, Ec2, MAAS, NoCloud, OpenNebula, OpenStack, OVF, SmartOS

The plan is to bring this back to 16.10 (Yakkety) in a Stable Release
Update [5] in the next few days.  This will expose it to more users than
the development release (Zesty), and then eventually make the change in
the 16.04 LTS (Xenial).

Scott

--
[1] Ubuntu cloud images are used as the basis for LXD, MAAS and public
    cloud images.
[2] https://cloud.google.com/compute/docs/instances/managing-instances
[3] http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/identify_ec2_instances.html
[4] https://bugs.launchpad.net/cloud-init/+filebug
[5] https://wiki.ubuntu.com/StableReleaseUpdates