TurnKey Linux's take on Ubuntu appliance development: KISS

[Liraz Siri]

Soren Hansen wrote:

>> We love packages, but an appliance is a higher-level concept and it's
>> important to understand that.
> 
> I realise that appliances are a higher-level concept. I just don't
> (yet?) see why that necessarily means that appliance-related
> customisations cannot live in the packages instead of being applied
> out-of-band.

Indeed, it just adds significantly to your development overhead. For a
good reason, changes to packaging infrastructure require more thought
than an ad-hoc configuration tweak by an administrator or appliance
builder.

Note that appliance developers and administrators are in the same boat
here. Even if it was possible I don't think it would be a good idea for
all possible system configuration and integration to go through debconf.

>> For example, let's compare the LAMP stack with the MySQL appliance.
> [...]
>> LAMP stack and the MySQL appliance have very similar components, but
>> different usage scenarios and hence - different configurations.
> 
> I don't see any reason why we couldn't ship both options in the mysql
> package. We could default to the current default and add a low priority
> debconf question asking which type you prefer and preseed that for the
> appliances. That way, all logic pertaining to mysql is contained in the
> mysql source package, but exposed to other entities for tweaking.
> 
> It takes a bit getting used to, but realising that /we/ (in the most
> inclusive sence) control the /entire/ software stack. We don't need to
> pursuade or argue with anyone but ourselves to make a change such as
> this.

But we must not forget that adding complexity has very real costs. If
Ubuntu and Debian had infinite resources perhaps this would not be an
issue, but they don't. Before you make far reaching changes to Ubuntu
packaging it would be wise to consider how those changes will be
regarded in Debian. This brings another cost into the fold - the cost of
trying to convince Debian maintainers to accept these changes or
alternatively the cost of maintaining a growing delta.

Administrator and appliance developers can integrate their systems
without having to discuss refinements to packaging policies or engage
many egos. Remember, avoiding this sort of friction was one of the
reasons Ubuntu was started as a separate project from Debian.

Worse, don't forget that anyone outside of the inner Ubuntu / Debian
circles is in a different position than you. It may feel to you like
/we/ would have an easy time controlling the entire stack, but it
doesn't feel that way to me. I can't even send messages to the list
without having to go through the moderation queue (usually that doesn't
seem to be a problem, but the original message that started this thread
was delayed for 2 weeks).

For an outsider it's much more fun and productive to spend time figuring
out integration details, cranking out appliances and getting immediate
feedback from users than messing around with packaging infrastructure,
negotiating with gatekeepers to get changes accepted and then waiting
for the next release cycle to see the fruits of your labor in the wild.

>> If you try to do this sort of thing with packages you are going to
>> either fail or be forced to add a tremendous amount of unnecessary
>> complexity at the package level.
> 
> Just because you extend the package to support these two options,
> doesn't mean that you have to extend it to support all sorts of odd
> variants in between these two extremes. It doesn't have to be any more
> complicated than what I outlined above.
> 
> Anything you have to patch out-of-band in an Ubuntu package for it to be
> useful, I would consider a bug. Within reason, of course.

Agreed. We avoid patching Ubuntu packages out-of-band.

>> Whenever a package is not supported in Ubuntu or worse, doesn't even
>> exist as a Debian package, that makes it much more difficult to create
>> and maintain appliances.
> 
> Definitely. For anything to be even considered an "official" Ubuntu
> appliance, it would have to consist only of software available as
> packages in the repository. It is the only way we can reasonably support
> anything.

Agreed. Unfortunately there is a great deal of excellent open source
software that is not officially in Debian or Ubuntu yet (e.g., Joomla,
Zimbra, Redmine, MindTouch Core, OpenBravo, ProjectPier). Originally we
avoided software that wasn't packaged but eventually we figured it was
better to have an imperfect appliance than no appliance at all.

Again, we're in the same boat as administrators here. It's much easier
to be able to install and update software via the package management
system, but when that's not an option we're going to find some other way
to install the software our users want.

> Have you usually found the dependencies of your software to be met, or
> do you usually have to install extra Perl, PHP, or Python modules to
> meet the needs of the software you're turning into an appliance?

Yes, we do occasionally need to add the extra module to satisfy
dependencies for non-packaged software. Especially with Ruby on Rails
applications which don't play well with the package manager.

Unfortunately, when the dependencies are too significant (e.g., try
installing Alfresco on Hardy) we've backed down from making an appliance
at all. Until the next version of Ubuntu LTS at least...

More often it's not dependencies we are installing but components which
are not yet available in Ubuntu/Debian but we feel add value to the
integration (e.g., Webmin, ShellInaBox, eXtplorer, etc.). In these cases
we package and maintain the software ourselves, and the packages live at
archive.turnkeylinux.org.

Cheers,
Liraz Siri
Cell: +972-54-2013512
Twitter: http://twitter.com/lirazsiri
Google Talk: liraz at turnkeylinux.org (Jabber/XMPP IM)