[DRAFT] Co-located services

[Gustavo Niemeyer]

Thanks for putting this in place Ben.

Some comments follow.

> Services such as logging, monitoring and storage often require some
> access to the runtime of the service they apply to. Under the current

We haven't debated storage in this context in the sprint, and it's
not clear the model of formula collocation fits well in it.  As we
covered recently here in the list, it feels like a broader problem that
we'll want to eventually cover internally in Ensemble as a first-class
feature.

My suggestion is that we keep this out of the spec for the moment.

> Co-located service/formula: A service designed for and deployed to the
> running container of another service.
> Parent service: The running service in whose service units we will be
> executing units of the co-located service.

Is it necessarily a parent/child relationship, or would it map better
to a peer style where both services are simply co-located with each
other?

> ensemble deploy <formula> [[--with <colo-formual [<service_name>]] ...]
>
> Each `with` stanza takes the name of a formula and an optional service
> name under which to deploy it as. The --with argument can be used

Optional parameters on arguments generally don't work very well in
practice. E.g.:

    ensemble deploy --with a b c

Is b a service name, or a formula name?

I also wonder a bit if we need special syntax for co-location.  It'd
be nice if we could find a way to have it just as a normal relation in
general, and it does feel like the problem being solved above is a
general issue: how to deploy services while establishing a relation.
This is the same kind of problem we'll face when we start to enforce
required relations, I believe.  If we solve that latter problem, we'll
solve co-located deployment syntax too.

> <y> and <z> will have their install and start hooks fired before <x>.
> This would ease services like storage into being properly setup within

What if we have multiple services co-located in the same container?
IMO we should restrict the difference of co-located relations to
containment, without imposing additional rules, and should try to
make both sides of the relation even (no parent-child relationship).
It makes them a lot closer to normal relations, and much easier
to understand and use (and develop! ;-).

> While the co-located services have relationships with the parent
> service (and the parent service may supply them with general
> information) its is generally considered a usage error for the parent
> service to ever query the co-located service's relation-data.

Why?  How's that different from a normal relation?

> The current model of reporting services should under-go some
> modifications. While evolving status is an evolved topic these options

Given the above points, it feels like the status might be pretty much
unchanged.

> The socket nested in the formula directory should allow each unit
> agent in the container the ability to communicate with the proper

Good point.

> Formula's YAML metadata file will grow new constructs. The first is a
> `co-located` top level flag which can be 'allow', 'always', 'never'

This section is diverging significantly from what we agreed in the sprint.
IIRC, the idea was simply to allow defining a relation as co-located. E.g.:

requires:
    syslog:
        interface: syslog-sync
        colocated: true

This would mean the services connected through this relation would
co-locate.  In addition to that, we'd introduce as a separate step the
concept of promiscuous relations.  This doesn't have to be done in a
first incarnation of the spec/feature, though.

Do we need anything else?

> The second risk is that this system is abused in the wild to escape
> possible limitations of the current system with regard to machine
> reuse. This could inadvertently allow for a new class of bad practice

Well put.  We need to enable multiple service units per machine before
colocation.

Thanks for starting the brainstorm on this Ben.

-- 
Gustavo Niemeyer
http://niemeyer.net
http://niemeyer.net/plus
http://niemeyer.net/twitter
http://niemeyer.net/blog

-- I never filed a patent.