Upstart helpers (abstract jobs and event aliases) for Oneiric

Scott James Remnant scott at netsplit.com
Tue Jun 7 16:43:59 UTC 2011


In CrOS, we do something similar. We have three "boot phase" jobs that most
other jobs are started and stopped by. They are:

  startup (distinct from the event)
  boot-services
  system-services

They are chained together thus:

  startup: start on startup (this has a main process that carries out jobs
similar to mountall)
  boot-services: start on stopped startup
  system-services: start on started boot-services

boot-services and system-services stay running until shutdown.

Other jobs then use:

  start on starting startup

  start on starting boot-services
  stop on stopping boot-services

  start on starting system-services
  stop on stopping system-services

The use of "ing" here is important; it means that "started boot-services"
means that all boot-services have started (since the starting event blocks
for them). So system-services happen entirely after boot-services (UI is a
boot service for us).

Scott

On Tue, Jun 7, 2011 at 7:00 AM, James Hunt <james.hunt at ubuntu.com> wrote:

> Hi All,
>
> This mail turned into rather an epic in the writing. The summary is I'm
> interested in thoughts on the two implementation options for "Proposal 2".
>
> Regards,
>
> James.
>
>
> = Overview =
>
> For a while Clint and I have thought it would make sense to introduce
> some "helper" events and jobs. The idea here being to:
>
> - Simplify existing .conf files.
> - Make it easier / speed up writing new .conf files.
> - Aid in understanding .conf files.
>
> The idea was discussed at UDS-O ([1]+[2]) and there were no fundamental
> issues
> raised so the plan is to introduce them for Oneiric.
>
> This mail explains the problem we're trying to address along with 2
> possible implementations.
>
> Let's start by describing the problem...
>
> = Problem Examples =
>
> == Example 1: plymouth.conf ==
>
> The first example is taken from /etc/init/plymouth.conf:
>
>  start on (starting mountall
>            or (runlevel [016]
>                and (stopped gdm
>                     or stopped kdm
>                     or stopped xdm
>                     or stopped lxdm
>                     or stopped lightdm
>                     or stopped uxlaunch)))
>
> === Observations ===
>
> 1) Overly complex "start on" condition.
>   - Indicative of refactoring requirement.
>     (since a rule of thumb is that most jobs should only
>      require ~3-4 events).
>   - Difficult to understand.
>   - Difficult to modify without breaking behaviour.
>
> 2) Cryptic runlevel condition.
>
> 3) Hard-coded list of Display Managers.
>
>   Bad since:
>
>   - Results in inability to handle new Display Managers efficiently.
>   - Promotes "cut-and-paste hell" (where the same bugs are pasted into
>     other .conf files and the problem is propagated, thus making it
>     more time-consuming to fix).
>
> == Example 2: gdm.conf ==
>
> Here is the upcoming /etc/init/gdm.conf:
>
>  start on ((filesystem
>              and (runlevel [!06]
>              and (started dbus
>              and (drm-device-added card0 PRIMARY_DEVICE_FOR_DISPLAY=1
>              or  (graphics-device-added PRIMARY_DEVICE_FOR_DISPLAY=1
>              or   stopped udevtrigger)))))
>              or  runlevel PREVLEVEL=S)
>
> === Observations ===
>
> 1) WT^H^H Errr... what?
>
>   You may be forgiven for dropping your Rich Tea biscuit into your cuppa
>   on seeing this for the first time (YMMV of course :)
>
> 2) Why are there two lines which appear to represent the same thing
>   (graphics device)?
>
> 3) Thought: surely there must be a simpler way to represent this condition?
>
> 4) Fragile.
>
> = The Problem(s) defined =
>
> - Too many job configuration files hard-code required *applications* for
>  which there are multiple alternatives.
>
>  What if you don't have that application installed? The Ubuntu archive
>  is huge and there are often many alternatives for system components.
>  So, by hard-coding applications, you risk breaking users who deviate
>  from the defaults, resulting potentially in a bad experience.
>
>  A level of abstraction would protect us from this problem.
>
>  Job Configuration Files should instead specify the *services* they
>  require.
>
> - Quite a few core Job Configuration Files have overly complex
>  conditions which can be abstracted.
>
> - Cut-and-pasting of complex conditions has led to problems.
>  We need to provide a way to avoid this sort of problem proliferating.
>
> = Proposals =
>
> == Proposal 1: Provide Event Aliases for Common Scenarios ==
>
> Create event aliases as shown below:
>
> |----------------+------------------|
> | Existing Event | Event Alias      |
> |----------------+------------------|
> | runlevel 0     | halt             |
> | runlevel 1     | single-user-mode |
> | runlevel S     | single-user-mode |
> | runlevel 2     | multi-user-mode  |
> | runlevel 6     | reboot           |
> | runlevel [016] | shutdown         |
> |----------------+------------------|
>
> == Proposal 2: Provide Abstract Jobs for Common Services ==
>
>
> |-----------------+-------------------------------------------------------------------|
> | Abstract Job    | Description
>           |
>
> |-----------------+-------------------------------------------------------------------|
> | display-manager | gdm, kdm, lightm, etc.
>            |
> | network-manager | NetworkManager, wicd, connman, etc
>            |
> | firewall        | ufw alias.
>            |
> | network         | started when *all* configured network interfaces and
> bridges "up" |
> | graphics-card   | starts when first graphics card added to system.
>            |
>
> |-----------------+-------------------------------------------------------------------|
>
> === Implementation ===
>
> There are two simple methods here we're considering.
>
> ==== Option 1 ====
>
> Update every package that provides a service such that its Job
> Configuration File sets and exports a "well-known" variable. The
> proposed list of environment variables which represent these services
> is:
>
>  DISPLAY_MANAGER
>  FIREWALL
>  GRAPHICS_CARD
>  NETWORK
>  NETWORK_MANAGER
>
> For example, each display manager package would be updated such that its
> .conf file specified:
>
>  env    DISPLAY_MANAGER=y
>  export DISPLAY_MANAGER
>
> Then, any job that requires a display manager could say:
>
>  start on starting DISPLAY_MANAGER=y
>
> ===== Observations =====
>
> - This might *appear* to be inefficient in that Upstart must check every
>  time *any* job starts to determine if DISPLAY_MANAGER=y is set in that
>  jobs environment. However, there is no additional overhead beyond how
>  Upstart currently works. Consider that...
>
>    start on starting gdm
>
>  ... is actually an alias for:
>
>    start on stating JOB=gdm
>
> - This would require *every* package that provides a service to be
>  updated. Admittedly this would only need to be done once though.
>
> - Might be confusing since users you *must* specify "=y" exactly
>  (dropping the "=y" won't work, and nor will specifying "=Y" (or "=1")).
>
> ==== Option 2 ====
>
> Create a Job Configuration File that hard-codes the list of known
> service providers. For example for "display-manger", we could have:
>
>  start on (starting gdm
>        or (starting kdm
>        or (starting lightdm
>        or (starting lxdm
>        or (starting slim
>        or (starting wdm
>        or  starting xdm))))))
>
>  stop on (stopping gdm
>        or (stopping kdm
>        or (stopping lightdm
>        or (stopping lxdm
>        or (stopping slim
>        or (stopping wdm
>        or  stopping xdm))))))
>
>  env    ABSTRACT_JOB=y
>  export ABSTRACT_JOB
>
> ===== Observations =====
>
> - Since this is an Abstract Job, it will have no PID, but this job will
>  "run" for the duration of the first display manager to be invoked.
>
> - We appear to have simply "moved the problem" since although we are no
>  longer hard-coding the list of display managers in "plymouth.conf", we
>  are hard-coding the list now in "display-manager.conf". However, we
>  have gained by doing this since:
>
>  - We have still created the level of abstraction desired.
>  - We have contained the problem into a single file: we define the list
>    of display managers *once*.
>  - We don't need to update every Ubuntu (and potentially every Debian)
>    package as would be required for "Option 1".
>  - We *could* conceivably auto-generate "display-manager.conf" from the
>    archive with a simple script which munged the output of:
>
>    apt-cache search x-display-manager|awk '{print $1}'|sort
>
> - The "ABSTRACT_JOB" variable would allow other jobs to detect that a
>  job was abstract (they shouldn't need to care, but just in case...)
>  This also avoids the unwieldliness of "abstract-display-manager" (or
>  even "abstract/display-manager" (were we to put the abstract jobs in
>  /etc/init/abstract/ say).
>
> ==== Personal Preference ====
>
> Although it looks rather ugly, my preference is currently for Option 2
> primarily since it would be quick and easy to modify the single source
> for each service. However, I could be persuaded otherwise :)
>
> = Rationale =
>
> By introducing Abstract Jobs and Event Alias "helpers", we can simplify
> the existing Job Configuration Files and make them more understandable.
> For example, the "start on" condition for "plymouth.conf" could become:
>
>  start on (starting mountall
>        or (shutdown and stopped display-manager))
>
> Similarly, "gdm.conf" could become the much simpler / easier to comprehend:
>
>  start on (filesystem and (started dbus and graphics-device-available)
>  stop on shutdown
>
> These changes may also incidentally minimise changes required by Ubuntu
> derivatives once the helpers become pervasive.
>
> = Documentation =
>
> We will of course ensure that all these helpers are documented
> appropriately and we plan to make
> maximum use of them to ease the work required for [2].
>
> -----
>
> [1] -
> https://blueprints.launchpad.net/ubuntu/+spec/foundations-o-upstart-convert-main-initd-to-jobs
> [2] -
> http://summit.ubuntu.com/uds-o/meeting/foundations-o-upstart-convert-main-initd-to-jobs/
>
>
> --
> James Hunt
>
> --
> upstart-devel mailing list
> upstart-devel at lists.ubuntu.com
> Modify settings or unsubscribe at:
> https://lists.ubuntu.com/mailman/listinfo/upstart-devel
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.ubuntu.com/archives/ubuntu-devel/attachments/20110607/f9a9c417/attachment-0001.html>


More information about the ubuntu-devel mailing list