[ubuntu-cloud] Fwd: #part-handler

[Scott Moser]

On Tue, 5 Mar 2013, Bo Shi wrote:

> Clint, Scott - sorry about the late reply.
>
> I did not explain myself particularly well.  Here's another attempt.
> The CloudInit documentation section for "Multipart Input" states "[a]
> single format of user data might not be enough to accomplish what you
> want. For example, you may want to insert an upstart job and also run
> a user-data script."  The example provided is:
>
>   $ ls
>   my-boothook.txt     my-include.txt      my-user-script.txt
>   my-cloudconfig.txt  my-upstart-job.txt
>
>   $ write-mime-multipart --output=combined-userdata.txt \
>      my-boothook.txt:text/cloud-boothook \
>      my-include.txt:text/x-include-url \
>      my-upstart-job.txt:text/upstart-job \
>      my-user-script.txt:text/x-shellscript \
>      my-cloudconfig.txt
>
> Is the order of execution of the various parts deterministic?  Can the
> order of execution be controlled in any manner, e.g. can I specify
> that "my-include.txt" depends on "my-user-script.txt" finishing?  A
> rather contrived use-case is if my-cloudconfig.txt was used to create
> some users and my-include.txt performed some operation on those users'
> home directories.  Another case is described in my original note.
>
> > To control order in which user-data is processed you should just need to
> > control the order in which it is created.
>
> Does this mean that the order in which the parts are specified in the
> invokation of "write-mime-multipart" above determines the order in
> which the parts gets executed?  Is execution serial (my observations
> lean toward no here)?

I guess I wasn't as clear as I could have been.
 a.) the order output by 'write-mime-multipart' is the same as the order
     of the command line arguments to it.  If you find otherwise, please
     let me know, thats a bug.
 b.) the order consumed by cloud-init's user-data handling is in-order
     that it appears in the user-data.  In your example above, cloud-init
     will process
       * 'my-boothook.txt'
       * then 'my-include.txt'
         * anything included by my-include.txt (and recursively)
       * my-upstart-job.txt
       * my-user-script.txt
       * my-cloudconfig.txt
 c.) "process" above is unfortunately not so clear.  Heres a guide:
    * archive formats (mime or cloud-config-archive or include) are
      exploded when found, and contents are processed at that point.
    * cloud-config-data is merged into the existing cloud-config in the
      order that it is seen.  The merge routine for cloud-config is very
      simplistic, and basically only supports overriding of fields.
      Josh Harlow has a branch at [1] to fix a feature request in bug
      1023179 to make this  more powerful.  I'm really looking forward to
      this.
        [1] https://code.launchpad.net/~harlowja/cloud-init/merge-types-differently/+merge/135310
    * boothook parts are executed immediately as they are processed.
      (cloud-config 'bootcmd' are processed slightly later)
    * upstart jobs are written to /etc/init immediately as they are
      processed.  And should the become ready for upstart usage
      immediately.
    * part-handlers are consumed immediately as they're seen, and then can
      immediately act on subsequent parts.
    * user-scripts are processed in the order they're seen, but the
    * processing just means shoving them into a file in
      /var/lib/cloud/instance/scripts (path from memory, so it could be
      wrong).  If the part has a filename, that filename is used.  If not,
      one is generated, and should increase.

      Then, later a 'runparts' equivalent is executed on that directory.
      runparts executes in C locale sorted order.

      The files that cloud-init created will be invoked in the order that
      they were written, but if you've provided filenames in the mime or
      cloud-config-archive parts then you can change that ordering.

      Also, 'runcmd' parts (from cloud-confing) are dropped into a file
      named 'runcmd' in that same directory.


That clear things up a bit?
Scott