<div dir="ltr"><div>[ Alright, I'm still unhappy about the level of details in this RFC, but I really need to send this out now; feel free to leave comments here or on:</div><a href="https://docs.google.com/document/d/1rzjvdlTFtHFds79UGl9LeIxSXxuGRhe9g-59QzqUf0U/edit">https://docs.google.com/document/d/1rzjvdlTFtHFds79UGl9LeIxSXxuGRhe9g-59QzqUf0U/edit</a> ]<div> </div><div><a href="https://lists.ubuntu.com/archives/snappy-devel/2015-January/000065.html" style="text-decoration:none">Introduced by Alexander on snappy-devel@</a>, Comfy was initially about bringing Ubuntu’s “comfort and goodness” to Snappy systems. This is an attempt to revisit this with an eye on comfortable Snappy experiences rather than the single Comfy universal use case from January. Over time, various expectations grew for Comfy, here are some sample use cases:<ol style="margin-top:0pt;margin-bottom:0pt"><li dir="ltr" style="list-style-type:decimal;font-size:14.6666666666667px;font-family:Arial;color:rgb(0,0,0);vertical-align:baseline;background-color:transparent">Get a more complete unix-ish / Ubuntu-ish experience on snappy devices; tools in Ubuntu standard/server, preferred/alternate shell, various utilities to get things done e.g. wget, lftp, vidir, dig, screen, tmux, file, various additional interpreters like perl</li><li dir="ltr" style="list-style-type:decimal;font-size:14.6666666666667px;font-family:Arial;color:rgb(0,0,0);vertical-align:baseline;background-color:transparent">Debugging/tuning/configuration at system level: access to logs, tcpdump, mtr, full editor for system config, wget etc.</li><li dir="ltr" style="list-style-type:decimal;font-size:14.6666666666667px;font-family:Arial;color:rgb(0,0,0);vertical-align:baseline;background-color:transparent">Development / software iterations: get/build/run/debug software on the target Snappy system</li><li dir="ltr" style="list-style-type:decimal;font-size:14.6666666666667px;font-family:Arial;color:rgb(0,0,0);vertical-align:baseline;background-color:transparent">Advanced debugging of snappy apps: gdb, strace, ltrace, access to logs, etc.</li></ol> There are fundamentally two classes of use cases:<ul style="margin-top:0pt;margin-bottom:0pt"><li dir="ltr" style="list-style-type:disc;font-size:14.6666666666667px;font-family:Arial;color:rgb(0,0,0);vertical-align:baseline;background-color:transparent">things that need to get done on Snappy systems now and are missing from the user experience: debugging a snappy app, configuring a snappy system comfortably, debugging a snappy system</li><li dir="ltr" style="list-style-type:disc;font-size:14.6666666666667px;font-family:Arial;color:rgb(0,0,0);vertical-align:baseline;background-color:transparent">things that are easily done on Ubuntu and can’t easily get done on Snappy systems today: development on the system, advanced unix user experience</li></ul> <h3 dir="ltr" style="line-height:1.38;margin-top:8pt;margin-bottom:0pt">Confortable development and debugging of snaps</h3> Snappy systems come with a different development experience from Ubuntu systems. Development isn’t done on the system anymore, instead the Snappy system is a target system and you develop from your Ubuntu Desktop host system (or perhaps even from other OSes like OSX or Windows). You might be running Ubuntu 14.04 and targeting Snappy 16.04. Developers shouldn’t have to login into the snappy system to develop their apps; instead, they should be able to get their app status, or run a debugger against their app, or get logs directly from the development environment, as found in other SDKs (Visual Studio, Eclipse, XCode etc.) for other OSes (Android/iOS/Windows…). We need to ensure a minimum bridge is available from Snappy to the development system and back:<ul style="margin-top:0pt;margin-bottom:0pt"><li dir="ltr" style="list-style-type:disc;font-size:14.6666666666667px;font-family:Arial;color:rgb(0,0,0);vertical-align:baseline;background-color:transparent">application status: are application’s services running as expected? (NB: this also came out in the <a href="https://docs.google.com/document/d/1DeR2JoCBDX2C1r6a2wV60uqvi7a89OwHh5gJ19tBZvM/edit" style="text-decoration:none">Snappy Shell</a> discussion)</li><li dir="ltr" style="list-style-type:disc;font-size:14.6666666666667px;font-family:Arial;color:rgb(0,0,0);vertical-align:baseline;background-color:transparent">application logs/output: did the application trigger any AppArmor denials? show its full stdout/stderr, show any related syslog output, show snappy startup messages for this app (think Android logcat), turn on more or less verbose output – particularly important for “printf debugging”</li><li dir="ltr" style="list-style-type:disc;font-size:14.6666666666667px;font-family:Arial;color:rgb(0,0,0);vertical-align:baseline;background-color:transparent">application data/state: access to application writable directory (check contents of its sqlite DB etc.)</li><li dir="ltr" style="list-style-type:disc;font-size:14.6666666666667px;font-family:Arial;color:rgb(0,0,0);vertical-align:baseline;background-color:transparent">application debugging: gdbserver based debugging or similar alternatives for other languages providing a debugger</li></ul> Concrete work to be done:<ul style="margin-top:0pt;margin-bottom:0pt"><li dir="ltr" style="list-style-type:disc;font-size:14.6666666666667px;font-family:Arial;color:rgb(0,0,0);vertical-align:baseline;background-color:transparent">design and add new snappy commands to get status of application’s services status, output, report interesting logs/issues etc.</li><li dir="ltr" style="list-style-type:disc;font-size:14.6666666666667px;font-family:Arial;color:rgb(0,0,0);vertical-align:baseline;background-color:transparent">add minimum amount of new snappy commands (for snappy-remote) and tools (like gdbserver) to core image for remote development to be possible</li></ul> <h3 dir="ltr" style="line-height:1.38;margin-top:8pt;margin-bottom:0pt">Providing an Ubuntu development environment</h3> While we do not want to encourage development on the device, we have to recognize that our tools are currently Ubuntu only, and often require a very recent version of Ubuntu. When people get started with Snappy, they might not be running the right Ubuntu version, or not even Ubuntu Linux or Linux at all. This means they often juggle between an Ubuntu Server SD card for their Raspberry Pi 2 and another one with Snappy Ubuntu Core, or use an Ubuntu Desktop virtual machine on their development host to run snappy commands, create images etc., copying large files back and forth. We should provide access to one or more standard environments for development of snaps directly on the device. This could be a docker or lxc container packaged as a regular snap (or just a docker/lxc image hosted in standard/official repositories). This environment would contain a nicely featured Ubuntu rootfs with development tools such as editors, utilities, docs etc. and the latest and correct versions of snappy tools. We could eventually provide multiple environments depending on the target language, e.g. a ruby/rails environment, a nodejs/web development, a java one etc. but at first it seems a single environment would be enough. It should be simple to build/install/run/debug snaps on the snappy system from this environment. This is similar to the <a href="https://coreos.com/docs/cluster-management/debugging/install-debugging-tools/" style="text-decoration:none">approach that CoreOS is following</a>. Concrete work to be done:<ul style="margin-top:0pt;margin-bottom:0pt"><li dir="ltr" style="list-style-type:disc;font-size:14.6666666666667px;font-family:Arial;color:rgb(0,0,0);vertical-align:baseline;background-color:transparent">create docker and/or lxc image(s) or snap(s) with a full blown development environment</li><li dir="ltr" style="list-style-type:disc;font-size:14.6666666666667px;font-family:Arial;color:rgb(0,0,0);vertical-align:baseline;background-color:transparent">add bridge between snappy system and this environment (e.g. /snappy/ to access snappy rootfs)</li></ul> <h3 dir="ltr" style="line-height:1.38;margin-top:8pt;margin-bottom:0pt">Comfortable system-level debugging and configuration</h3> Some tools like tcpdump, strace, top etc. would make no sense to run in a container with its own process and network namespaces and limited access to the host system. Instead, these should run at the top level. But how do we provide these tools? These need to work even if a snap isn’t updated along the rootfs or vice-versa, these need to find their data and libraries in the right place and these need to interact with the right environment (e.g. see the right processes, binaries for debugging etc.). <div dir="ltr" style="margin-left:0pt"><table style="border:none;border-collapse:collapse"><colgroup><col width="163"><col width="437"></colgroup><tbody><tr style="height:0px"><td style="border:1px solid rgb(0,0,0);vertical-align:top;padding:7px">1. Add to current rootfs</td><td style="border:1px solid rgb(0,0,0);vertical-align:top;padding:7px">of course, these could simply be added to the Snappy Ubuntu Core rootfs eating space on all installations. Pros: easy, works immediately.Cons: space or default rootfs.</td></tr><tr style="height:0px"><td style="border:1px solid rgb(0,0,0);vertical-align:top;padding:7px">2. Alternate rootfs</td><td style="border:1px solid rgb(0,0,0);vertical-align:top;padding:7px">we could provide an alternate rootfs snap, with a lot of tools installed. It probably won’t be possible to switch between rootfes on an installed device (due to e.g. modified config files in writable-path with passwd/group changes), so developers would end up having to test their apps on two rootfses as to make sure they don’t rely on the debug/devel/comfortable rootfs. Pros: easy to create.Cons: more testing for us and for app devs, less unicity, can’t switch back and forth.</td></tr><tr style="height:0px"><td style="border:1px solid rgb(0,0,0);vertical-align:top;padding:7px">3. Comfy snap with relocatable tools</td><td style="border:1px solid rgb(0,0,0);vertical-align:top;padding:7px">we could provide a bunch of tools as a snap. It would contain shared libraries for these tools or a static version of them, and a mechanism for the tools to find their data based on their location. e.g. “file” would be a wrapper or a patched version that would know how to find magic data to identify file types under /apps/comfy.ubuntu/current/… This could be implemented by patching tools with chrpath or similar. Pros: nice to use.Cons: a lot of work to create, work specific to each tool required.</td></tr><tr style="height:0px"><td style="border:1px solid rgb(0,0,0);vertical-align:top;padding:7px">4. Relocate snappy core fs</td><td style="border:1px solid rgb(0,0,0);vertical-align:top;padding:7px">move snappy ubuntu core libs and tools out of standard pathes as to allow mounting overlays. e.g. move /bin/sh to /snappy/bin/sh, and the same for ld-linux, libc etc. then provide an overlay with /lib/libc, ld-linux, strace etc. This overlay could be a regular snap with rootfs data (system-data/), or handled specially with union mounts when starting an admin shell. In this scenario, the Snappy Ubuntu Core rootfs could be based on busybox rather than trying to relocate all our base tools and be quite minimal or we could patch tools with chrpath and similar. Pros: easy to add tools and libraries to extra snap, could be used for other snaps/use cases.Cons: intrusive for snappy rootfs; could be a lot of work. This design would be visible to snappy apps (e.g. which ld-linux would they use etc.).</td></tr></tbody></table></div> Approaches not considered here: LD_PRELOAD based approaches have drawbacks:<ul style="margin-top:0pt;margin-bottom:0pt"><li dir="ltr" style="list-style-type:disc;font-size:14.6666666666667px;font-family:Arial;color:rgb(0,0,0);vertical-align:baseline;background-color:transparent">these break suid binaries and ptrace</li><li dir="ltr" style="list-style-type:disc;font-size:14.6666666666667px;font-family:Arial;color:rgb(0,0,0);vertical-align:baseline;background-color:transparent">these would provide confusing names (e.g. you’d see long filenames)</li></ul> Concrete work to be done:<ul style="margin-top:0pt;margin-bottom:0pt"><li dir="ltr" style="list-style-type:disc;font-size:14.6666666666667px;font-family:Arial;color:rgb(0,0,0);vertical-align:baseline;background-color:transparent">select an approach</li><li dir="ltr" style="list-style-type:disc;font-size:14.6666666666667px;font-family:Arial;color:rgb(0,0,0);vertical-align:baseline;background-color:transparent">create a proof of concept and iterate</li></ul><div> </div>-- </div><div>Loïc Minier </div></div>