In CrOS, we do something similar. We have three "boot phase" jobs that most other jobs are started and stopped by. They are:<div> </div><div> startup (distinct from the event)</div><div> boot-services</div><div> system-services</div><div> </div><div>They are chained together thus:</div><div> </div><div> startup: start on startup (this has a main process that carries out jobs similar to mountall)</div><div> boot-services: start on stopped startup</div> <div> system-services: start on started boot-services</div><div> </div><div>boot-services and system-services stay running until shutdown.</div><div> </div><div>Other jobs then use:</div><div> </div><div> start on starting startup</div> <div> </div><div> start on starting boot-services</div><div> stop on stopping boot-services</div><div> </div><div> start on starting system-services</div><div> stop on stopping system-services</div><div> </div> <div>The use of "ing" here is important; it means that "started boot-services" means that all boot-services have started (since the starting event blocks for them). So system-services happen entirely after boot-services (UI is a boot service for us).</div> <div> </div><div>Scott <div class="gmail_quote">On Tue, Jun 7, 2011 at 7:00 AM, James Hunt <<a href="mailto:james.hunt@ubuntu.com">james.hunt@ubuntu.com</a>> wrote: <blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex;"> Hi All, This mail turned into rather an epic in the writing. The summary is I'm interested in thoughts on the two implementation options for "Proposal 2". Regards, James. = Overview = For a while Clint and I have thought it would make sense to introduce some "helper" events and jobs. The idea here being to: - Simplify existing .conf files. - Make it easier / speed up writing new .conf files. - Aid in understanding .conf files. The idea was discussed at UDS-O ([1]+[2]) and there were no fundamental issues raised so the plan is to introduce them for Oneiric. This mail explains the problem we're trying to address along with 2 possible implementations. Let's start by describing the problem... = Problem Examples = == Example 1: plymouth.conf == The first example is taken from /etc/init/plymouth.conf: start on (starting mountall or (runlevel [016] and (stopped gdm or stopped kdm or stopped xdm or stopped lxdm or stopped lightdm or stopped uxlaunch))) === Observations === 1) Overly complex "start on" condition. - Indicative of refactoring requirement. (since a rule of thumb is that most jobs should only require ~3-4 events). - Difficult to understand. - Difficult to modify without breaking behaviour. 2) Cryptic runlevel condition. 3) Hard-coded list of Display Managers. Bad since: - Results in inability to handle new Display Managers efficiently. - Promotes "cut-and-paste hell" (where the same bugs are pasted into other .conf files and the problem is propagated, thus making it more time-consuming to fix). == Example 2: gdm.conf == Here is the upcoming /etc/init/gdm.conf: start on ((filesystem and (runlevel [!06] and (started dbus and (drm-device-added card0 PRIMARY_DEVICE_FOR_DISPLAY=1 or (graphics-device-added PRIMARY_DEVICE_FOR_DISPLAY=1 or stopped udevtrigger))))) or runlevel PREVLEVEL=S) === Observations === 1) WT^H^H Errr... what? You may be forgiven for dropping your Rich Tea biscuit into your cuppa on seeing this for the first time (YMMV of course :) 2) Why are there two lines which appear to represent the same thing (graphics device)? 3) Thought: surely there must be a simpler way to represent this condition? 4) Fragile. = The Problem(s) defined = - Too many job configuration files hard-code required *applications* for which there are multiple alternatives. What if you don't have that application installed? The Ubuntu archive is huge and there are often many alternatives for system components. So, by hard-coding applications, you risk breaking users who deviate from the defaults, resulting potentially in a bad experience. A level of abstraction would protect us from this problem. Job Configuration Files should instead specify the *services* they require. - Quite a few core Job Configuration Files have overly complex conditions which can be abstracted. - Cut-and-pasting of complex conditions has led to problems. We need to provide a way to avoid this sort of problem proliferating. = Proposals = == Proposal 1: Provide Event Aliases for Common Scenarios == Create event aliases as shown below: |----------------+------------------| | Existing Event | Event Alias | |----------------+------------------| | runlevel 0 | halt | | runlevel 1 | single-user-mode | | runlevel S | single-user-mode | | runlevel 2 | multi-user-mode | | runlevel 6 | reboot | | runlevel [016] | shutdown | |----------------+------------------| == Proposal 2: Provide Abstract Jobs for Common Services == |-----------------+-------------------------------------------------------------------| | Abstract Job | Description | |-----------------+-------------------------------------------------------------------| | display-manager | gdm, kdm, lightm, etc. | | network-manager | NetworkManager, wicd, connman, etc | | firewall | ufw alias. | | network | started when *all* configured network interfaces and bridges "up" | | graphics-card | starts when first graphics card added to system. | |-----------------+-------------------------------------------------------------------| === Implementation === There are two simple methods here we're considering. ==== Option 1 ==== Update every package that provides a service such that its Job Configuration File sets and exports a "well-known" variable. The proposed list of environment variables which represent these services is: DISPLAY_MANAGER FIREWALL GRAPHICS_CARD NETWORK NETWORK_MANAGER For example, each display manager package would be updated such that its .conf file specified: env DISPLAY_MANAGER=y export DISPLAY_MANAGER Then, any job that requires a display manager could say: start on starting DISPLAY_MANAGER=y ===== Observations ===== - This might *appear* to be inefficient in that Upstart must check every time *any* job starts to determine if DISPLAY_MANAGER=y is set in that jobs environment. However, there is no additional overhead beyond how Upstart currently works. Consider that... start on starting gdm ... is actually an alias for: start on stating JOB=gdm - This would require *every* package that provides a service to be updated. Admittedly this would only need to be done once though. - Might be confusing since users you *must* specify "=y" exactly (dropping the "=y" won't work, and nor will specifying "=Y" (or "=1")). ==== Option 2 ==== Create a Job Configuration File that hard-codes the list of known service providers. For example for "display-manger", we could have: start on (starting gdm or (starting kdm or (starting lightdm or (starting lxdm or (starting slim or (starting wdm or starting xdm)))))) stop on (stopping gdm or (stopping kdm or (stopping lightdm or (stopping lxdm or (stopping slim or (stopping wdm or stopping xdm)))))) env ABSTRACT_JOB=y export ABSTRACT_JOB ===== Observations ===== - Since this is an Abstract Job, it will have no PID, but this job will "run" for the duration of the first display manager to be invoked. - We appear to have simply "moved the problem" since although we are no longer hard-coding the list of display managers in "plymouth.conf", we are hard-coding the list now in "display-manager.conf". However, we have gained by doing this since: - We have still created the level of abstraction desired. - We have contained the problem into a single file: we define the list of display managers *once*. - We don't need to update every Ubuntu (and potentially every Debian) package as would be required for "Option 1". - We *could* conceivably auto-generate "display-manager.conf" from the archive with a simple script which munged the output of: apt-cache search x-display-manager|awk '{print $1}'|sort - The "ABSTRACT_JOB" variable would allow other jobs to detect that a job was abstract (they shouldn't need to care, but just in case...) This also avoids the unwieldliness of "abstract-display-manager" (or even "abstract/display-manager" (were we to put the abstract jobs in /etc/init/abstract/ say). ==== Personal Preference ==== Although it looks rather ugly, my preference is currently for Option 2 primarily since it would be quick and easy to modify the single source for each service. However, I could be persuaded otherwise :) = Rationale = By introducing Abstract Jobs and Event Alias "helpers", we can simplify the existing Job Configuration Files and make them more understandable. For example, the "start on" condition for "plymouth.conf" could become: start on (starting mountall or (shutdown and stopped display-manager)) Similarly, "gdm.conf" could become the much simpler / easier to comprehend: start on (filesystem and (started dbus and graphics-device-available) stop on shutdown These changes may also incidentally minimise changes required by Ubuntu derivatives once the helpers become pervasive. = Documentation = We will of course ensure that all these helpers are documented appropriately and we plan to make maximum use of them to ease the work required for [2]. ----- [1] - <a href="https://blueprints.launchpad.net/ubuntu/+spec/foundations-o-upstart-convert-main-initd-to-jobs" target="_blank">https://blueprints.launchpad.net/ubuntu/+spec/foundations-o-upstart-convert-main-initd-to-jobs</a> [2] - <a href="http://summit.ubuntu.com/uds-o/meeting/foundations-o-upstart-convert-main-initd-to-jobs/" target="_blank">http://summit.ubuntu.com/uds-o/meeting/foundations-o-upstart-convert-main-initd-to-jobs/</a> -- James Hunt -- upstart-devel mailing list <a href="mailto:upstart-devel@lists.ubuntu.com">upstart-devel@lists.ubuntu.com</a> Modify settings or unsubscribe at: <a href="https://lists.ubuntu.com/mailman/listinfo/upstart-devel" target="_blank">https://lists.ubuntu.com/mailman/listinfo/upstart-devel</a> </blockquote></div> </div>