Determine status of events in complex job start condition

[Dimitri John Ledkov]

On 18 February 2015 at 14:10, Owen Dunn <osd1000 at cam.ac.uk> wrote:
> Hello.  I originally asked this question of cjwatson, but he sent me to
> #upstart whose topic sent me onwards here :-)
>
> I'm trying to determine why an Upstart job with a complex start condition
> isn't starting.  I've booted with --verbose so I can see the events in
> syslog, but ideally I'd like to ask Upstart which events the job was still
> waiting for.
>

Upstart can serialise it's state into json, that is exported as DBus api.

(same serialisation is used to preserve and restore stack upon re-exec)

Thus at appropriate time, you can call GetState to get the json dump -
that will list all blocking events and which ones have not been
satisfied for a particular job.

http://people.canonical.com/~jhunt/upstart/scripts/get_state.sh is the
sample usage.


> To use a concrete example, I have /etc/init/lightdm.override:
>
> start on ((filesystem
>            and runlevel [!06]
>            and started dbus
>            and plymouth-ready
>            and maths-onboot-done)
>           or runlevel PREVLEVEL=S)
>
> so that lightdm is only started after our local onboot procedure has
> finished.  Sometimes this works, but sometimes it doesn't and I'd like to

It is best to do reverse, block lightdm startup until you are finished.

1) keep lightdm job unmodified
2) create a new job "block-lightdm.conf"
start on starting lightdm
stop on maths-onboot-done
task
exec sleep 1000

Thus when lightdm start on condition is satisfied - "starting lightdm"
will be emitted. At which point, "block-lightdm.conf" will be
starting. If maths-onboot-done event arrives or 1000 seconds lapse,
block-lightdm will bail out and lightdm will come up.

A good example on how to pre-empt /all/ graphical login manager's in
Ubuntu and execute certain type of work before them is demonstrated in
Ubiquity (Ubuntu Installer).
It pre-empts graphical login manager and displays installer interface,
if that crashes or exits one gets dropped into lightdm, which on live
cd is configured to auto-login into live desktop session.

http://bazaar.launchpad.net/~ubuntu-installer/ubiquity/trunk/view/head:/debian/ubiquity.ubiquity.upstart

start on (starting gdm
 or starting kdm
 or starting xdm
 or starting lxdm
 or starting lightdm)
stop on (runlevel [06]
or stopping gdm
or stopping kdm
or stopping xdm
or stopping lxdm
or stopping lightdm)

task

exec /usr/share/ubiquity/start-ubiquity-dm

> work out why.  initctl status lightdm just says lightdm stop/waiting, and
> there appears to be no way to ask `what events has the lightdm start
> condition already received'.
>
> How long do jobs hang on to events that are in `and' clauses like this? Is
> there any timeframe within which all the events have to be received?
>

all events must be received to start the job. A classic example is:
$ cat job.conf
start on A and B

$ initctl emit A
$ initctl emit B
// job starts
// events clear
$ initctl stop job
$ initctl emit A
// job is waiting for event B

A job will wait for it's start on condition to be satisfied, fully,
indefinitely.

> Is there anything that resets the accumulation of events?
>

starting the job instance, or stopping it. There is another one -
configuration is not replaced for the "in-flight" jobs

config is read // start on (A and B)

event A arrives // job instance is created

config is changed, and re-read // start on A

At this point the "in-flight" job instance retains its config and
blocks on receiving event B, as in-flight config is never replaced. If
one subsequently does $ stop A, it will drop waiting  & drop old
config & drop the instance. The updated config (start on A) will then
be promoted to be the active one and thus subsequent "emit A" will
start that job.


> In particular one of the things our local onboot procedure does is to modify
> /etc/init/lightdm.override and other Upstart job files.  Would modifying a
> job config file reset the list of events it had received?
>

Depends when such modification is done. If you are doing it during
startup, it will be racy. As above. If you do it e.g. at install-time
/ initramfs-time / pre upstart exec, you should be fine.

-- 
Regards,

Dimitri.