TurnKey Linux's take on Ubuntu appliance development: KISS

[Liraz Siri]

Carlos Ribeiro wrote:
>     There's a rule against Debian packages messing around with the
>     configuration files of other packages and that rule exists for a good
>     reason.
> 
>     If you break the rules, then a franken-package could theoretically you
>     could do anything, we just don't believe that's the right tool for
>     the job.
> 
> I'm still not sure if this is as big a deal is it may seem. The package
> itself would not alter any system-level configuration file during its
> installation. It would provide a script that needs to be explictily
> invoked by the user, where it would (more or less automatically) tune
> the config files. It's not that much different than webmin or any other
> system configuration panel, with the exception that it's done automatically.

So you are suggesting that a package would contain a tool that
automatically modifies system-level configurations with input from the
user. You know I think something like that would be a good tool to have
for package development in general. There are lots of scenarios where it
would be useful for a package to be able to modify system configuration
- something that's currently not allowed. I'd like to have a tool like
in that in my belt and if it works reliably enough it could help with
appliance development as well. Sure.

>     Packages are meant to be modular. Unfortunately, there's no reliable way
>     to merge system level configurations. For example, MySQL can either
>     accept connections from remote hosts or not accept connections. What's
>     the result of a merge? You could say, in that case let's just say that
>     those appliances conflict like we do with packages. Trouble is, an
>     appliance can have a ton of these configurations, each of which can be
>     incompatible with the configurations of any another appliance, or even
>     worse, incompatible with the result of a combination of appliances.
> 
> First of all, I think that appliances should be specialized. It should
> not be possible to install two appliances on the same base system, in
> the same way we can't install Apache in different modes on the same
> system. They should be mutually exclusive packages. If you need an
> appliance to do two different things (for example, DNS and webserver on
> the same box), than either use two appliances or develop a combined one
> from the scratch.

I agree and it doesn't sound like any one who knows what they're talking
about is arguing that point so at least that is in consensus: no merging
of appliances.

> Also, merging is already a problem even when we simply manually edit the
> config files. I see that as a limitation of the packaging system, and it
> happens whenever one tries to upgrade/dist-upgrade a heavily customized
> system. Some packages are already much smarter in this regard than
> others (for instance, it's a long time since I have any trouble with
> Apache2) - but others always give me trouble.
> 
> So it may be the case that the original packages should be improved and
> get smarter at handling upgrades. Breaking up config files as Apache
> does (inside a config dir) is a great way to do it as it allows the user
> to put all his changes in separate files that will not be touched upon
> during an upgrade.There may be other issues to look at but it should be
> handled on a case by case basis.

Absolutely agreed. This is a general issue and I for one would like to
see more packages moving towards modular configuration files with the
help of upstream. Unfortunately, we're not there yet and in some areas
the problem is getting worse with projects using monolithic XML-based
configuration formats or database driven configurations.

> In this sense I completely agree. But on the other hand I think that a
> long term project to make it possible to use packages to the same effect
> would be a laudable goal, given the following criteria:
> 
> 1) the appliance meta-package defines the dependencies;
> 
> 2) all appliance meta-packages provide the same functionality (let's
> say, "appliance") and are mutually exclusive;
> 
> 3) the package itself does not alter any system configuration, it just
> allows the user to do it easily with some extra information and a few
> keypresses (or clicks if it's web-based);
> 
> 4) all required packages should be able to handle merging automatically
> during a upgrade, not touching the custom configuration done either by
> the user or the appliance configuration script;
> 
> 5) dependencies should be configured in such a way that base packages
> can only be upgraded when the appliance meta-package gets upgraded too.
> The side effect is that the publisher of the appliance meta-package
> would need to take responsibility at testing and issuing new versions
> whenever a major upgrade or security patch hits its base packages.
> 
> I'm not saying that it's easy or doable right now - I just think that
> it's possible and worth a try. But of course, you folks have much more
> experience tha I have at it, and I may be not aware of all the issues.
> What do you think?

Great ideas but as you mention a bit out of the scope of what we have
the resources to accomplish at the moment.

So I'm thinking the community's direction should be to improve packaging
infrastructure generically with the help of upstream and new
experimental innovations to configuration management and in a few years
when most of the major gotchas have been addressed re-evaluate and
refine the technical direction for appliance development to include
these advances. As a developer I'd love to see that happen but at the
same time I realize most users don't really care about what happens
under the hood. They just want stuff to work. Right now. TurnKey is all
about bridging the gap between the value the best open source software
can deliver and the value end-users are getting.

I'm not saying the back-end stuff isn't important just that we're trying
to be pragmatic and invest our limited resources where it helps users
the most.

Cheers,
Liraz Siri
Cell: +972-54-2013512
Twitter: http://twitter.com/lirazsiri
Google Talk: liraz at turnkeylinux.org (Jabber/XMPP IM)