Dependencies

[Sean E. Russell]

On Sunday 08 October 2006 01:58, Jerry Haltom wrote:
> Sure. But when do you CLEAR it? Job maintains a list of... all waited

You are right... this is the part of my suggestion that requires serious 
design consideration.

> state. What if somebody mounts and then unmounts a filesystem really
> fast? You start operating on the "mount" event when the system is

If the "mount" event was the last event in the requirements list, then the 
requirements are met and you start the script.  Then when the unmount occurs, 
you clear the mount flag.  If the script has registered itself to be stopped 
on an unmount, then you also (at that time) stop the script.  

How is this ANY different than any other script that is started and stopped 
only by mount events?

> already unmounted. All the scripts which deal with this stuff are going
> to have to properly check conditions before they execute anyways.
...
> Countered above, I believe. Still, when all 4 events are received in
> full, all 4 conditions still need to be checked right then.

I don't agree.  But even if the script did check conditions, it would only 
have to check the conditions ONCE, rather than every time one of the events 
in the requirements list came in.

> I think with some reusable and smart shell scripts, each job can take
> care of that, without much overhead.

I was really hoping the solution wouldn't involve another functions.sh hack.

Quoting you out of order, to short-circuit any flame wars:

> I just made this idea up right now, btw. Not sure if that's a good idea
> at all. I do like the way it's isolated to the job code though.

I'm making all of my stuff up on the fly, too, so if I'm misunderstanding your 
implementation and finding problems where there aren't any, please don't take 
offense.

> start on event1
> start on event2
> start on event3
>
> script
>    source /usr/share/upstart/functions
>    upstart_waitfor event1 event2 event3
>    # check conditions, do stuff.
> end script
>
> In this example, waitfor waits for all of event1, event2 and event3 to
> arrive. It creates a variable to track progress, adds UPSTART_EVENT (the
> initiating event) to it, and opens a socket to upstart to track the
> rest.

Creates a variable where?  Upstart is ignorant of multiple event dependencies, 
and would fork this script three times, unless you're implying a change to 
Upstart to prevent this.  Each script evaluation would have its own stack 
space (I really hope that scripts aren't sharing variable space), so all 
three scripts would be waiting for all three events.  When the final event 
comes it, there'd be a cascade of evaluations of the rest of the script, 
which will certainly screw up the system.

> Basically, while waiting, it results in a sh instance sitting around
> blocking on a socket. Not exactly much overhead. "waitfor" could timeout
> and fail if it wanted to, too.

Don't you mean one sh process per event in the deps list?  Doesn't this seem 
sloppy, and wasteful, to you?  I still don't see how you're going to avoid 
having three (in your example) separate instances of the script competing to 
start the process.

The only alternative implementation I can think you're thinking of is that 
upstart_waitfor() actually writes some state into a temp file somewhere, and 
checks that, exiting the script unless all conditions are met.  This would 
avoid the "three sh processes waiting and competing" problem... but writing 
temp files?  Yuck.  It sounds like a recipe for a race condition, or an 
locking mechanism (which tend to be relatively complex and error-prone).

The first implementation, IMHO, is broken; the second smells like a hack to 
get around deficiencies in the underlying init mechanism.

> > > Cups is a hard one. I'd say it should either watch the network on it's
> >
> > Well, Cups was just an example.  The Gentoo init scripts are full of
> > examples of services which have conditional requirements.  The idea is
> > that there's a mechanism for saying "if this service is going to be
> > started, then wait for it to start.  Otherwise, just start now."
>
> I'm not sure if we've addressed this, except to propose a "before"
> argument to a job. Basically just orders the jobs an event will effect.
> Could actually be done the same way. Both jobs are started equally by
> upstart, one job waits for the other to be started before finishing.

Hm.  More implementation details, please.  I'm trying to think how you could 
do this.

You can't do it in the script, because the script has no knowledge of whether 
the other service will ever be started (it *is* conditional, after all).  You 
could do it in Upstart, because upstart knows whether a particular service is 
available to receive requests.  However, each script would have to register 
such conditional dependencies with Upstart.  Example:

service1:

	start on filesystem/started
	uses service2
	...

service2:

	start on network/started
	...

If service2 doesn't exist in the event listener queue, Upstart does nothing 
but start service1 when event1 occurs.  However, if service2 DOES exist (EG, 
in /etc/event.d), then Upstart would add a multiple event dependency to 
service1, so that, effectively, service1's event dependencies would look like 
this:

	start on filesystem/started service2/started

Again, this would be much easier to do in Upstart than in scripts.  Each 
script would have to be able to query Upstart about its state, parse which 
services were available, register additional listener events with Upstart... 
it really sounds like a lot of extra work to do merely to avoid a small 
amount of extra work in the Upstart init process.

Cheers!

-- 
### SER   
### Deutsch|Esperanto|Francaise|Linux|XML|Java|Ruby|Aikido|Iaido
### http://www.ser1.net  jabber.com:ser  ICQ:83578737 
### GPG: http://www.ser1.net/Security/ser_public.gpg