App installer design: click packages: 0install

[Thomas Leonard]

[ I thought I sent this already, but I didn't see it appear, so 
resending a slightly updated version ]

On 2013-05-15 03:47, Barry Warsaw wrote:
> On May 14, 2013, at 02:37 PM, Colin Watson wrote:
>
>>>> Barry spent
>>>> some time looking at 0install, and it wasn't too bad a fit but we would
>>>> still need to solve many of the same system integration problems.
>>>
>>> Is there a list of these issues somewhere?
>>
>> Barry Warsaw would have to speak to that, I think.  But mainly, there's
>> no way for the system to declare any kind of integration facilities at
>> all, as far as I can see, so we'd have to solve that problem anyway.
>
> What I think Colin is getting at here is that any system is going to have to
> deal with declaring the files that allow an app to interoperate with system
> services.  Think along the lines of dbus .service files, .desktop files, etc.
> These will be things we might know about right now (such as the above), but
> they maybe be additional integration points in the future.

So the idea here is that packages can't specify their own .desktop or 
.service files directly, since that would allow them to escape any 
sandboxing or otherwise make suprising changes to the system behaviour. 
Instead, the installer creates these files in response to user actions.

For example, when a user adds a program using the 0desktop GUI, 0desktop 
creates a .desktop file to launch the application. Hints in the metadata 
help it do this (for example, the <needs-terminal/> element indicates 
that the .desktop file should launch it via the user's preferred 
terminal emulator). Since the format is XML, custom elements and 
attributes can be added for new cases without changing 0install itself.

But this is just my personal preference. You are always free to scan the 
installed apps' metadata looking for hooks, if you feel that won't 
create security issues.

For example, if the user tries to open a file, you could scan the 
installed apps to see which can handle that MIME type. But you'd have to 
confirm the choice with the user, at least the first time, since 
otherwise any random application could register itself to open any file.

> Let's say for example, some future system service comes with a new version of
> the SDK or OS image, that allows apps to define push notifications, by
> including a .push file that has to get installed somewhere to be registered.
> The app itself will only declare that it has a myapp.push, and you won't have
> anything like a maintainer script to install myapp.push.  Instead, the
> system's push notifier will have to be responsible for registering a plugin
> with the installer system to say "okay, now when you see .push files, put them
> here and do this something to register it."
>
> You don't want the logic for that in the installer, and you don't want it in
> the app.  You want it as part of the system push notifier.  This latter will
> register a trigger that gets run whenever the installer sees that an app has
> declared a .push file *and* you want it to run over all app .push files when
> the system push notifier is installed.  In the latter case, imagine that you
> have a dozen apps that support push notifications installed on your phone, but
> the system push notifier hasn't been installed yet.  Once it is, all your
> already installed push-enabled apps will Just Work.
>
> This all means that the installer must have plugins for handling new
> functionality and the declarative manifest must be extensible, i.e. the
> installer will ignore extensions that it doesn't (yet) have a registered
> plugin for.

If you wanted to avoid plugins, you could define a fixed naming scheme, 
so that if APP provides HOOK then the installer creates an entry in e.g.

   <config>/hooks/HOOK/APP.hook

Then the push service could look in <config>/hooks/push/*.hook to find 
all apps that support it quickly. You could then just add all hooks, 
whether the service using them is installed or not.

For hooks with security implications, it might be better to wait until 
the user requests the feature. An annoyance of Android is that you can 
only grant permissions at install time, so every application asks for 
every possible permission up front.

> I didn't get far enough on 0install (or don't remember ;) whether it has that
> capability.
>
>> I will certainly concede that the rest of 0install is very flexible and
>> has a usefully pluggable design; you seem to have done a good job on it
>> and Barry spoke well of it.  It just seems that there's a bit of a
>> mismatch in that it's kind of over the top in terms of flexibility for
>> what we need and seems to be entirely missing some other things we need,
>> as above; and it would be a lot of code to rewrite in some other
>> language if (as is possible, though not imminent) we decide that Python
>> is just too heavyweight to run routinely on the Ubuntu phone.

On the installing side, you're always going to be limited by the network 
rather than the CPU. You might save a little power with a more efficient 
language, but I doubt it's very much.

For running apps, you'd probably want to replace the Python with 
something faster. When you do

   0install run foo

it loads .config/0install.net/apps/foo/selections.xml, sets the 
environment variables and launches the app. That doesn't need to be 
Python and ideally it wouldn't be. We use Python because it makes 
0install easy to use on a wide range of platforms, even where 
distribution support is missing.

One final thought here: you don't necessarily need to run the solver on 
the phone anyway. You could run a central store which selects the 
default versions of each program and library and just serves those 
selections to the phone.

e.g.

A developer provides a 0install app called http://.../someapp. You want 
this in your store. On your server, you do:

   0install download http://.../someapp --xml > someapp-selections.xml

When a user wants this app on their phone, you just transfer 
someapp-selections.xml to the phone, which then requests any missing 
packages (from your mirror server probably). From time to time, you 
update the selections on your server.

Here, 0install provides an automated way to get software from developers 
to your store rather than all the way to the phone.
You'd probably still want 0install on the phone in case the user wanted 
to run 3rd-party apps or use different versions, but it wouldn't be used 
in the default case. It would nice if there were an easy way to swich 
between Ubuntu-recommended versions and upstream/custom versions.

> On just few other points:
>
> I'm not really a big fan of XML formats, since they're not human consumable
> (readable or writable).  Others may disagree, which is fine, but I don't have
> any interest in arguing about it.  Probably not a deal breaker, since the XML
> files can always be generated, either by the 0install tools or something that
> takes a JSON or YAML as input, etc.

One of the nice things about XML is that it can be extended in an 
obvious way by third-party tools, e.g. to define new attributes without 
messing up existing tools.

> I liked that it was fairly easy to wrap executables in "security" containers,
> but I didn't dig into this in a lot of detail.
>
> It seemed to have good security considerations.
>
> IIRC, there's no built-in support for uninstallation.

   0install destroy APP

will remove the launchers for APP[*], but doesn't automatically evict 
the software from the cache. "0store manage" lets you remove things 
individually, but for a phone you'd want the system to GC packages after 
an app has been destroyed. I think the Sugar desktop environment does 
this, and the Windows version has something similar too.

It's pretty easy (~/.config/0install.net/apps/*/selections.xml lists all 
the packages and libraries you need, so delete anything that's in the 
cache but not referenced there), but so far I've never run out of disk 
space so I haven't implemented it.

[*] 0desktop predates this and currently has a separate destroy feature. 
At some point, "0desktop" and "0install add" will be unified.

> We probably want something like an online-only search REST API for finding
> apps, and then downloading from the URL such a search gives us.  Thus, there's
> no cleverness in the repository layout, nor on the device, for finding apps.

"0install search QUERY" does that (since 0install 2.1), but the search 
engine at the server end isn't very clever at the moment. You'd probably 
want to run your own back-end.

> Anyway, that's about all the additional stuff I've gleaned from my previously
> private experiments with 0install that hasn't already been discussed in this
> thread.
>
> -Barry
>
> P.S. If we *were* to accept that the installer could be written in Python, it
> would *have* to be Python 3.  No more Python 2 on the phone, please!

0install runs with Python 2 or Python 3. The Debian package defaults to 
using Python 2 because it's faster and the API is more stable (i.e. less 
chance of breakage when a new Python release comes out).

BTW, if using Python 3, it also (optionally) supports the new Tulip 
event loop, in case you want to drop the GObject dependency.


-- 
Dr Thomas Leonard        http://0install.net/
GPG: 9242 9807 C985 3C07 44A6  8B9A AE07 8280 59A5 3CC1
GPG: DA98 25AE CAD0 8975 7CDA  BD8E 0713 3F96 CA74 D8BA