Upstart 1.0 Design discussion

[Casey Dahlin]

On 09/14/2009 09:21 AM, Scott James Remnant wrote:
>> I'll save my follow-up for LPC. It may take some intense explanation.
>> The crux of it is that we might be able to implement a few more stanzas
>> that don't outwardly make sense, and, with the addition of being able to
>> #include other files within job definitions, be able to eliminate large
>> quantities of upstart's code in favor of a job state machine described
>> in job definition syntax. Freaky huh?
>>
> I think it's a good idea to write things down on the mailing list,
> rather than save them for LPC - even if the in-person stuff goes
> different directions, it means those on the ML get a preview of what
> you're thinking of ;-)
>
> (Also the very process of writing a mail tends to help focus the
> thoughts)
>
> Scott
>

Well, since you say so, and since I promised a brain-dump today...

Lets suppose an extremely simplified version of upstart. It has "Jobs" 
as you describe them, but jobs are either "up" or "down," that is they 
don't have these starting/started/stopping/stopped events/states.

They do have while stanzas, which relate them to other jobs, and on 
stanzas which let them react to events. They emit two events, one when 
they come up, and one when they go down.

Lets also, for a moment, rip out all the service management. For the 
moment, Jobs aren't jobs. They just hold a condition, and that's it. 
Forget for just a second about interacting with processes.

Now, lets add a series of seemingly useless features, and then I'll 
perform a magic trick with them.

First we'll add the implements stanza: lets require every job to have a 
stanza called implements. It looks like this:

implements service

Every job must have one, and only one. The majority of them will have 
"service" as the argument. What does this do? It looks for a file in 
/etc/init/types/ called service.conf, and bodily includes it at the top 
of our job definition file. Inside of the included file, it expands the 
string @job to be the name of the job it was included in. Weird? Yes. 
I'm going to set this aside and come back to it later, but let me point 
out one thing: this stanza, whatever its use, would be reeeeeely easy to 
implement. Dirt cheap.

Lets add another, stranger one. "job." It looks like this:

job foo
     on bar
     when baz
end job

This lets us create other jobs inside of a job definition. The new job 
is a "sub-job" of the job we define. So, for example: If in a file 
"boing.conf" we had the above stanza, there would be a new job called 
"boing foo". Note that this doesn't adjust the state of the job at all: 
"boing" can be down, and "foo" can be up. Also notice we use whitespace 
as a namespace separator, rather than "boing.foo" or "boing:foo".

Its weird, but again, reeeeeely easy and cheap to implement.

Lets try one more:

somename exec /bin/action
some_othername script
   /bin/action
end script

These are action definitions. They associate a script with a variable in 
the job (called somename, or some_othername in this case). We can make 
them start like this:

spawn somename on fooevent
kill somename TERM on fooevent

Spawn simply executes the script stored at "somename" and leaves. It 
doesn't monitor it or tie it to the job in anyway. However, events will 
be emitted while somename is running at interesting times (forking, 
dieing, etc). Matching one would look like this:

on foojob somename SIGTERM

Kill does what you'd expect. It sends a signal to the process if running.

One more caveat, if you omit the name, a name of _default is assumed, so

_default exec /bin/action

does the same thing as

exec /bin/action

Its an odd, gutted sort of return of the service management 
functionality, but it comes without any internal state machines, just 
trivial notifications, which are not obviously useful, but, again, 
reeeeely cheap to implement.

So the magic trick: remember that service.conf file? Lets put some stuff 
in it.

Hey y'all, watch this:

spawn _default on @job starting
kill _default TERM on @job stopping

job starting
	from @job up until (@job running up or @job stopping up or
				@job down)
	on @job up or @stopping down
end job

job running
	from @job _default forked until (@job stopping or
		@job _default exited)
end job

job stopping
	from @job up until @job starting
	on @job running down
end job

So we cut out most of upstart, added 3 things that are all reeeeeeely 
cheap to implement, and got most of what we cut out back. Perfect? Not 
yet, but interesting...

--CJD