App installer design: click packages

[Colin Watson]

On Thu, May 09, 2013 at 09:41:26AM +0100, Thomas Leonard wrote:
> On 2013-05-08 11:14, Colin Watson wrote:
> >So, at Steve Langasek's request, I've been putting together a proof of
> >concept of a low-level app package installer and packaging format.
> >Highlights of what it can do so far are:
> >
> >  * no dependencies between apps; single implicit dependency on the base
> >    system by way of a Click-Base-System field
> 
> 0install packages can depend on debs if they want to, so if you
> create a virtual deb package representing the Ubuntu base system
> then you could just depend on that (e.g. "requires Ubuntu >= 13.4").
> If you want to be cross platform, it would make more sense to
> specify individual dependencies, though.

This is a deliberate constraint to avoid interaction with the (large)
system package database, rather than a feature that was too hard to
implement.  Is it easy to disable this feature in 0install and do
something more like the system that Listaller and click-package use of
depending on system frameworks declared in text files?

(I suppose it's not critical; the primary requirement is bulletproof
reliability, and performance is merely an important secondary
requirement.)

> >  * base package manager overhead, i.e. the time required to install a
> >    trivial package containing a single small file, is about 0.15 seconds
> >    on a newish x86 laptop and about 0.6 seconds on a Nexus 7 (and that's
> >    with the current prototype implementation in Python; a later
> >    implementation could be in C and would then be faster still)
> 
> 0.109 seconds here on my laptop (including the time to download the
> archive from Apache running on localhost; normally network delays
> will dominate, of course).

debs install pretty quickly on my laptop too, but querying the package
database is going to be slow on typical phone hardware, so I'd still
like to make sure we avoid that.

> >  * not limited to installing as root, although there may be similar
> >    constraints elsewhere to ensure that apps can't edit their own code
> >    at run-time
> 
> Same. If system-wide sharing is enabled then downloads go under
> /var/cache and so apps cannot edit their own code. Without
> system-wide sharing, the files are stored in the user's home
> directory. They're read-only, but Linux doesn't currently provide us
> with a way to stop an app running as a user from changing that
> user's files. We're looking into using the new user namespaces
> support for sandboxing.

I think we will have some degree of sharing but not to the extent where
one user can force another to upgrade, so we'll be running the installer
as a separate user.

> >  * strawman design for hooks into system packages, which will be
> >    entirely declarative from the app's point of view
> 
> What does this mean?

You mean "entirely declarative from the app's point of view"?  I mean
that the app contains no code executed with the permissions of the
package installer, although system packages may contain code that is
executed with those permissions when apps install certain kinds of files
or declare certain kinds of metadata.

> >Obvious items I still need to work on:
> >
> >  * produce a strawman hooks implementation with some real worked
> >    examples
> >  * integrate (or demonstrate how to integrate) the container isolation
> >    properties worked on elsewhere
> 
> Would be interested to hear more about this.

https://click-package.readthedocs.org/en/latest/hooks.html is the short
answer for now.

> >Is there anything else people can think of that a system like this needs
> >to consider?
> 
> Package signing, key distribution and trust, mirroring,

All of these belong in a higher-level tool.

> dependencies,

Intentionally excluded.

> roll-back, parallel installation of stable and testing
> versions,

Different versions are unpacked into different directories, so these are
easy.  (In fact the main thing to do is the converse: decide when to
garbage-collect old versions.)

> checking for updates,

Higher-level tool.

> source packages, ...? ;-)

I made a conscious choice not to do anything like Debian source
packages.  I have no problem with them personally, but they're a
consistent source of confusion among app developers, and of course (ugh)
there is the inevitable confusion for distributors of proprietary apps.
However, I'm conscious that it's important to make it easy for people to
distribute free software the right way.  This is an open question for
now and I've put it on the to-do list.

> > Barry spent
> >some time looking at 0install, and it wasn't too bad a fit but we would
> >still need to solve many of the same system integration problems.
> 
> Is there a list of these issues somewhere?

Barry Warsaw would have to speak to that, I think.  But mainly, there's
no way for the system to declare any kind of integration facilities at
all, as far as I can see, so we'd have to solve that problem anyway.

I will certainly concede that the rest of 0install is very flexible and
has a usefully pluggable design; you seem to have done a good job on it
and Barry spoke well of it.  It just seems that there's a bit of a
mismatch in that it's kind of over the top in terms of flexibility for
what we need and seems to be entirely missing some other things we need,
as above; and it would be a lot of code to rewrite in some other
language if (as is possible, though not imminent) we decide that Python
is just too heavyweight to run routinely on the Ubuntu phone.

Thanks,

-- 
Colin Watson                                       [cjwatson at ubuntu.com]