Upstart helpers (abstract jobs and event aliases) for Oneiric

James Hunt james.hunt at
Tue Jun 7 14:00:03 UTC 2011

Hi All,

This mail turned into rather an epic in the writing. The summary is I'm
interested in thoughts on the two implementation options for "Proposal 2".



= Overview =

For a while Clint and I have thought it would make sense to introduce
some "helper" events and jobs. The idea here being to:

- Simplify existing .conf files.
- Make it easier / speed up writing new .conf files.
- Aid in understanding .conf files.

The idea was discussed at UDS-O ([1]+[2]) and there were no fundamental issues
raised so the plan is to introduce them for Oneiric.

This mail explains the problem we're trying to address along with 2
possible implementations.

Let's start by describing the problem...

= Problem Examples =

== Example 1: plymouth.conf ==

The first example is taken from /etc/init/plymouth.conf:

  start on (starting mountall
            or (runlevel [016]
                and (stopped gdm
                     or stopped kdm
                     or stopped xdm
                     or stopped lxdm
                     or stopped lightdm
                     or stopped uxlaunch)))

=== Observations ===

1) Overly complex "start on" condition.
   - Indicative of refactoring requirement.
     (since a rule of thumb is that most jobs should only
      require ~3-4 events).
   - Difficult to understand.
   - Difficult to modify without breaking behaviour.

2) Cryptic runlevel condition.

3) Hard-coded list of Display Managers.

   Bad since:

   - Results in inability to handle new Display Managers efficiently.
   - Promotes "cut-and-paste hell" (where the same bugs are pasted into
     other .conf files and the problem is propagated, thus making it
     more time-consuming to fix).

== Example 2: gdm.conf ==

Here is the upcoming /etc/init/gdm.conf:

  start on ((filesystem
              and (runlevel [!06]
              and (started dbus
              and (drm-device-added card0 PRIMARY_DEVICE_FOR_DISPLAY=1
              or  (graphics-device-added PRIMARY_DEVICE_FOR_DISPLAY=1
              or   stopped udevtrigger)))))
              or  runlevel PREVLEVEL=S)

=== Observations ===

1) WT^H^H Errr... what?

   You may be forgiven for dropping your Rich Tea biscuit into your cuppa
   on seeing this for the first time (YMMV of course :)

2) Why are there two lines which appear to represent the same thing
   (graphics device)?

3) Thought: surely there must be a simpler way to represent this condition?

4) Fragile.

= The Problem(s) defined =

- Too many job configuration files hard-code required *applications* for
  which there are multiple alternatives.

  What if you don't have that application installed? The Ubuntu archive
  is huge and there are often many alternatives for system components.
  So, by hard-coding applications, you risk breaking users who deviate
  from the defaults, resulting potentially in a bad experience.

  A level of abstraction would protect us from this problem.

  Job Configuration Files should instead specify the *services* they

- Quite a few core Job Configuration Files have overly complex
  conditions which can be abstracted.

- Cut-and-pasting of complex conditions has led to problems.
  We need to provide a way to avoid this sort of problem proliferating.

= Proposals =

== Proposal 1: Provide Event Aliases for Common Scenarios ==

Create event aliases as shown below:

| Existing Event | Event Alias      |
| runlevel 0     | halt             |
| runlevel 1     | single-user-mode |
| runlevel S     | single-user-mode |
| runlevel 2     | multi-user-mode  |
| runlevel 6     | reboot           |
| runlevel [016] | shutdown         |

== Proposal 2: Provide Abstract Jobs for Common Services ==

| Abstract Job    | Description                                                       |
| display-manager | gdm, kdm, lightm, etc.                                            |
| network-manager | NetworkManager, wicd, connman, etc                                |
| firewall        | ufw alias.                                                        |
| network         | started when *all* configured network interfaces and bridges "up" |
| graphics-card   | starts when first graphics card added to system.                  |

=== Implementation ===

There are two simple methods here we're considering.

==== Option 1 ====

Update every package that provides a service such that its Job
Configuration File sets and exports a "well-known" variable. The
proposed list of environment variables which represent these services


For example, each display manager package would be updated such that its
.conf file specified:


Then, any job that requires a display manager could say:

  start on starting DISPLAY_MANAGER=y

===== Observations =====

- This might *appear* to be inefficient in that Upstart must check every
  time *any* job starts to determine if DISPLAY_MANAGER=y is set in that
  jobs environment. However, there is no additional overhead beyond how
  Upstart currently works. Consider that...

    start on starting gdm

  ... is actually an alias for:

    start on stating JOB=gdm

- This would require *every* package that provides a service to be
  updated. Admittedly this would only need to be done once though.

- Might be confusing since users you *must* specify "=y" exactly
  (dropping the "=y" won't work, and nor will specifying "=Y" (or "=1")).

==== Option 2 ====

Create a Job Configuration File that hard-codes the list of known
service providers. For example for "display-manger", we could have:

  start on (starting gdm
        or (starting kdm
        or (starting lightdm
        or (starting lxdm
        or (starting slim
        or (starting wdm
        or  starting xdm))))))

  stop on (stopping gdm
        or (stopping kdm
        or (stopping lightdm
        or (stopping lxdm
        or (stopping slim
        or (stopping wdm
        or  stopping xdm))))))

  env    ABSTRACT_JOB=y

===== Observations =====

- Since this is an Abstract Job, it will have no PID, but this job will
  "run" for the duration of the first display manager to be invoked.

- We appear to have simply "moved the problem" since although we are no
  longer hard-coding the list of display managers in "plymouth.conf", we
  are hard-coding the list now in "display-manager.conf". However, we
  have gained by doing this since:

  - We have still created the level of abstraction desired.
  - We have contained the problem into a single file: we define the list
    of display managers *once*.
  - We don't need to update every Ubuntu (and potentially every Debian)
    package as would be required for "Option 1".
  - We *could* conceivably auto-generate "display-manager.conf" from the
    archive with a simple script which munged the output of:

    apt-cache search x-display-manager|awk '{print $1}'|sort

- The "ABSTRACT_JOB" variable would allow other jobs to detect that a
  job was abstract (they shouldn't need to care, but just in case...)
  This also avoids the unwieldliness of "abstract-display-manager" (or
  even "abstract/display-manager" (were we to put the abstract jobs in
  /etc/init/abstract/ say).

==== Personal Preference ====

Although it looks rather ugly, my preference is currently for Option 2
primarily since it would be quick and easy to modify the single source
for each service. However, I could be persuaded otherwise :)

= Rationale =

By introducing Abstract Jobs and Event Alias "helpers", we can simplify
the existing Job Configuration Files and make them more understandable.
For example, the "start on" condition for "plymouth.conf" could become:

  start on (starting mountall
        or (shutdown and stopped display-manager))

Similarly, "gdm.conf" could become the much simpler / easier to comprehend:

  start on (filesystem and (started dbus and graphics-device-available)
  stop on shutdown

These changes may also incidentally minimise changes required by Ubuntu
derivatives once the helpers become pervasive.

= Documentation =

We will of course ensure that all these helpers are documented appropriately and we plan to make
maximum use of them to ease the work required for [2].


[1] -
[2] -

James Hunt

More information about the ubuntu-devel mailing list