RFC: Comfortable Snappy (aka Comfy ;-)

Sun Jun 21 16:10:09 UTC 2015

[ Alright, I'm still unhappy about the level of details in this RFC, but I
really need to send this out now; feel free to leave comments here or on:
https://docs.google.com/document/d/1rzjvdlTFtHFds79UGl9LeIxSXxuGRhe9g-59QzqUf0U/edit
 ]

Introduced by Alexander on snappy-devel@
<https://lists.ubuntu.com/archives/snappy-devel/2015-January/000065.html>,
Comfy was initially about bringing Ubuntu’s “comfort and goodness” to
Snappy systems. This is an attempt to revisit this with an eye on
comfortable Snappy experiences rather than the single Comfy universal use
case from January.

Over time, various expectations grew for Comfy, here are some sample use
cases:

   1.

   Get a more complete unix-ish / Ubuntu-ish experience on snappy devices;
   tools in Ubuntu standard/server, preferred/alternate shell, various
   utilities to get things done e.g. wget, lftp, vidir, dig, screen, tmux,
   file, various additional interpreters like perl
   2.

   Debugging/tuning/configuration at system level: access to logs, tcpdump,
   mtr, full editor for system config, wget etc.
   3.

   Development / software iterations: get/build/run/debug software on the
   target Snappy system
   4.

   Advanced debugging of snappy apps: gdb, strace, ltrace, access to logs,
   etc.

There are fundamentally two classes of use cases:

   -

   things that need to get done on Snappy systems now and are missing from
   the user experience: debugging a snappy app, configuring a snappy system
   comfortably, debugging a snappy system
   -

   things that are easily done on Ubuntu and can’t easily get done on
   Snappy systems today: development on the system, advanced unix user
   experience

Confortable development and debugging of snaps

Snappy systems come with a different development experience from Ubuntu
systems. Development isn’t done on the system anymore, instead the Snappy
system is a target system and you develop from your Ubuntu Desktop host
system (or perhaps even from other OSes like OSX or Windows). You might be
running Ubuntu 14.04 and targeting Snappy 16.04.

Developers shouldn’t have to login into the snappy system to develop their
apps; instead, they should be able to get their app status, or run a
debugger against their app, or get logs directly from the development
environment, as found in other SDKs (Visual Studio, Eclipse, XCode etc.)
for other OSes (Android/iOS/Windows…).

We need to ensure a minimum bridge is available from Snappy to the
development system and back:

   -

   application status: are application’s services running as expected? (NB:
   this also came out in the Snappy Shell
   <https://docs.google.com/document/d/1DeR2JoCBDX2C1r6a2wV60uqvi7a89OwHh5gJ19tBZvM/edit>
   discussion)
   -

   application logs/output: did the application trigger any AppArmor
   denials? show its full stdout/stderr, show any related syslog output, show
   snappy startup messages for this app (think Android logcat), turn on more
   or less verbose output – particularly important for “printf debugging”
   -

   application data/state: access to application writable directory (check
   contents of its sqlite DB etc.)
   -

   application debugging: gdbserver based debugging or similar alternatives
   for other languages providing a debugger

Concrete work to be done:

   -

   design and add new snappy commands to get status of application’s
   services status, output, report interesting logs/issues etc.
   -

   add minimum amount of new snappy commands (for snappy-remote) and tools
   (like gdbserver) to core image for remote development to be possible

Providing an Ubuntu development environment

While we do not want to encourage development on the device, we have to
recognize that our tools are currently Ubuntu only, and often require a
very recent version of Ubuntu. When people get started with Snappy, they
might not be running the right Ubuntu version, or not even Ubuntu Linux or
Linux at all. This means they often juggle between an Ubuntu Server SD card
for their Raspberry Pi 2 and another one with Snappy Ubuntu Core, or use an
Ubuntu Desktop virtual machine on their development host to run snappy
commands, create images etc., copying large files back and forth.

We should provide access to one or more standard environments for
development of snaps directly on the device. This could be a docker or lxc
container packaged as a regular snap (or just a docker/lxc image hosted in
standard/official repositories). This environment would contain a nicely
featured Ubuntu rootfs with development tools such as editors, utilities,
docs etc. and the latest and correct versions of snappy tools.

We could eventually provide multiple environments depending on the target
language, e.g. a ruby/rails environment, a nodejs/web development, a java
one etc. but at first it seems a single environment would be enough. It
should be simple to build/install/run/debug snaps on the snappy system from
this environment.

This is similar to the approach that CoreOS is following
<https://coreos.com/docs/cluster-management/debugging/install-debugging-tools/>
.

Concrete work to be done:

   -

   create docker and/or lxc image(s) or snap(s) with a full blown
   development environment
   -

   add bridge between snappy system and this environment (e.g. /snappy/ to
   access snappy rootfs)

Comfortable system-level debugging and configuration

Some tools like tcpdump, strace, top etc. would make no sense to run in a
container with its own process and network namespaces and limited access to
the host system. Instead, these should run at the top level. But how do we
provide these tools? These need to work even if a snap isn’t updated along
the rootfs or vice-versa, these need to find their data and libraries in
the right place and these need to interact with the right environment (e.g.
see the right processes, binaries for debugging etc.).

1. Add to current rootfs

of course, these could simply be added to the Snappy Ubuntu Core rootfs
eating space on all installations.

Pros: easy, works immediately.

Cons: space or default rootfs.

2. Alternate rootfs

we could provide an alternate rootfs snap, with a lot of tools installed.
It probably won’t be possible to switch between rootfes on an installed
device (due to e.g. modified config files in writable-path with
passwd/group changes), so developers would end up having to test their apps
on two rootfses as to make sure they don’t rely on the
debug/devel/comfortable rootfs.

Pros: easy to create.

Cons: more testing for us and for app devs, less unicity, can’t switch back
and forth.

3. Comfy snap with relocatable tools

we could provide a bunch of tools as a snap. It would contain shared
libraries for these tools or a static version of them, and a mechanism for
the tools to find their data based on their location. e.g. “file” would be
a wrapper or a patched version that would know how to find magic data to
identify file types under /apps/comfy.ubuntu/current/… This could be
implemented by patching tools with chrpath or similar.

Pros: nice to use.

Cons: a lot of work to create, work specific to each tool required.

4. Relocate snappy core fs

move snappy ubuntu core libs and tools out of standard pathes as to allow
mounting overlays. e.g. move /bin/sh to /snappy/bin/sh, and the same for
ld-linux, libc etc. then provide an overlay with /lib/libc, ld-linux,
strace etc. This overlay could be a regular snap with rootfs data
(system-data/), or handled specially with union mounts when starting an
admin shell. In this scenario, the Snappy Ubuntu Core rootfs could be based
on busybox rather than trying to relocate all our base tools and be quite
minimal or we could patch tools with chrpath and similar.

Pros: easy to add tools and libraries to extra snap, could be used for
other snaps/use cases.

Cons: intrusive for snappy rootfs; could be a lot of work. This design
would be visible to snappy apps (e.g. which ld-linux would they use etc.).

Approaches not considered here:

LD_PRELOAD based approaches have drawbacks:

   -

   these break suid binaries and ptrace
   -

   these would provide confusing names (e.g. you’d see long filenames)

Concrete work to be done:

   -

   select an approach
   -

   create a proof of concept and iterate

-- 
Loïc Minier
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.ubuntu.com/archives/snappy-devel/attachments/20150621/5b7c02f9/attachment-0001.html>