Upstart config file generation

[Clint Byrum]

Excerpts from Jan Ceuleers's message of Sun May 01 10:48:14 -0700 2011:
> Hi there.
> 
> I'd like to pick your brains, if I may. There are applications whose 
> dependencies on hardware and other applications depend on their 
> configuration. This means that it is not possible for packagers of these 
> applications to provide a generic upstart configuration file.
> 
> Two possibilities for dealing with this: we either regard those 
> applications as broken, or we should create a framework (or at least 
> some features) enabling applications or packager glue scripts to 
> generate upstart configuration files.
> 
> It's perhaps easier to discuss this if I provide an example. The backend 
> part of MythTV [1] has at least two kinds of dependencies which depend 
> on its own configuration: tuners (i.e. video capture devices) and mysql.
> 
> If the backend starts before the tuners are fully initialised (which can 
> take a long time, with firmware loading, oscillator calibration and 
> whatnot) it will, as of version 0.24, start OK except that it won't be 
> able to make any recordings. So it won't fail to start, and there won't 
> be any opportunity for upstart to retry starting it. Please refer to [2] 
> for a discussion of what is needed in the start on stanza in order to 
> cater for this.

In this case, the tuner intialization should emit an event that can
generically be used by anything requiring them. Either do it in an
upstart task:

# initialize-tuners
start on startup
task
exec /usr/sbin/initialize-tuners

Then you can just 'start on started initialize-tuners' (or would it be
stopped... I keep forgetting how the blocking works w/ tasks)

Or if its handled by udev (as it sounds like it should be) then just
have the things that need it

start on tuner-device-added

(see man upstart-udev-bridge for more information).

If there's something outside of udev that runs the initialization,
it can emit its own event:

initctl --no-wait emit tuners-initialized

So the service could do

start on tuners-initialized and X and Y

> 
> If the backend starts before mysql is available then it will fail, and 
> upstart does have an opportunity to retry. No harm done then. Not ideal 
> either, because this attempt will have needlessly slowed the boot 
> procedure down. So why not put the dependency in the default (and 
> static) upstart configuration file? Because in the general case the 
> database need not reside on the same server as the backend (e.g. in the 
> case of a slave backend).

I'm working on a new wait-for-state.conf (which I hope to upload this
coming week to Oneiric, and will submit upstream for inclusion) which
will make it this easy:

if [ -x /usr/sbin/mysqld ] && [ -f /etc/init/mysql.conf ] ; then
  start wait-for-state WAIT_FOR=mysql WAIT_STATE=started WAITER=mythtv WAIT_FOREVER=1
fi

You would put that in the pre-start for your script which can use its
normal start on without mentioning mysql. It will only try to start
mysql if it is installed, and will fail if mysql fails to start.

Another thought is socket activation, but mysql may take many minutes
to start up so its not a great candidate.

> 
> So what do you think: are applications that require (or would benefit 
> from) generated upstart config files broken, or should something be done 
> about this? In the latter case, is this something that upstart should 
> (help) solve, or should it be left wholly to other tools to address? Or 
> is it just not that important a problem?

Upstart has the mechanisms its just new and thus not clear how to use
them.  I don't think we need auto-generated jobs, just more generic jobs.