RFC: Comfy - a CLI framework to bring the Ubuntu cloud feeling to snappy Ubuntu Core

Tue Jan 13 23:28:04 UTC 2015

On Tue, Jan 13, 2015 at 6:51 AM, Alexander Sack <asac at canonical.com> wrote:
> On Tue, Jan 13, 2015 at 1:11 PM, Martin Pitt <martin.pitt at ubuntu.com> wrote:
>> Hey all,
>>
>>> we have been asked a few times about adding one or another tool to
>>> ubuntu-core to help you get your job done.
>>
>> I think this needs a more concrete definition. I take it we really
>> don't want to try and build a framework for development, as that would
>> need a gazillion packages, GUI, etc. For those cases it would be much
>> better to make it easy to push stuff back and forth between a regular
>> Ubuntu desktop and a snappy VM (or physical machine), which to me
>> smells more like an sshfs or 9p thing than trying to put an entire
>> development desktop into snappy.
>
> Agreed. Dev is a special scenario that needs to be tackled separately.
>
>>
>> By the looks of your list it sounds like you aim for a comfortable
>> shell environment aimed at admins? I. e. you need to be able to poke
>> around and see what's wrong with your snappy packages/services, but
>> you don't need to build/develop packages on it?
>
> Correct, that's the current proposed way of thinking about comfy!
>
>>
>>
>> Alexander Sack [2015-01-13 12:32 +0100]:
>>>     + ?: accountsservice - discuss if this is useful/needed
>>
>> This is only really useful for graphical desktops, so "N" IMHO.
>
> OK, unless someone disagrees I will mark that as N for now.
>
>>
>>>     + !!: apport -> prob. needs to be in ubuntu-core for crash reporting?
>>
>> For crashes in ubuntu-core itself it might be useful, but only in the
>> restricted "phone" mode -- i. e. auto-submitting crashes to
>> errors.u.c., not Launchpad bugs (as these need a web browser and
>> interactivity).
>
> Right; maybe we should see if we can make a mini apport for just crash
> submission? Is it worthwhile wrt to disk footprint etc.?
>
>>
>>>     + Y: busybox-static
>>
>> I think this would only make sense as part of a "rescue" mode. But our
>> boot never breaks, right? :-)
>
> I assumed there was good reason why we have this in our cloud default
> install and couldn't see the diff why we would need it there and not
> here.
>
> Anyone from cloud world can comment if and why we have that in our cloud image?

I'm not positive, but I think it's used when (re-)building the cloud-initramfs.

>>
>>>     + N: command-not-found - highly apt specific (asac believes); double check
>>
>> Right, it tells you which apt packages you need to install for a
>> command. As you cannot do that on snappy anyway, I'd leave it out.

Or, rather, it would be nice if command-not-found learned how to
support snappy and snap packages.  "Oh, you're running Snappy and
you're trying to use the docker command -- you're in luck!  We have a
snap package for that.  Just snap install docker!"

>>
>>>     + Y: cryptsetup
>>>     + Y: cryptsetup-bin
>>
>> If we want to support encrypted devices, this belongs into snappy
>> itself, not comfy.
>
> We just yesterday discussed a related topic and Steve brought up that
> at least root encryption won't be theoretically needed as the content
> will be the immutable, public ubuntu-core. I assume having crypted
> writable partitions etc. makes surely sense though. If cryptsetup is
> needed for runtime mounting such rather than after install admin bits,
> then I agree that if we do it it should move to ubuntu-core.

Agree that the base OS itself is all public, baseline, immutable information.

What might be useful, though, is using encryption with overlayroot,
which we've been able to do reliable in our cloud images for a few
years now with the overlayroot package:

http://blog.dustinkirkland.com/2012/08/introducing-overlayroot-overlayfs.html

This is how we could ensure that any private, locally generated data
gets encrypted.  (Key management is a whole other question, as usual.)

> Is that the case?
>
>>
>>>     + Y: debconf-i18n
>>
>> IMHO we don't want debconf and thus this package. You can't apt on
>> snappy anyway, and pretty much the only thing that would make use of
>> this is "dpkg-reconfigure tzdata" for selecting a different time zone.
>> But you can also do that with other CLI tools.
>
> oops. Yes, this was an oversight for sure. Didn't mean to put debconf
> parts in here. Well spotted; changed to N.
>
>>
>>>     + Y: dmidecode
>>
>> Rather obsolete and unnecessary IMHO.
>
> Anyone disagrees with kicking it out? for now marking it as ? with pitts comment

I don't disagree, but I wasn't aware that dmidecode was that deprecated?

>>
>>>     + Y: ed
>>
>> Urgh, how could we forget about this :-)
>> (Seriously, what for?)
>
> i didnt see how the requirement of cloud image would be different from
> snappy. But yes, unless this is a dependency we might really consider
> to get rid of this. I will mark it accordingly as D now.
>
>>
>>>     + Y: eject
>>
>> This should be !! or N IMHO.
>
> Yes, let's see if anyone objects making this N
>
>>
>>>     + Y: ethtool
>>
>> Smells more like !!
>
> Anyone else feels ethtool is so essential that it should be in
> ubuntu-core itself?

The one thing that I've found ethtool essential for, from time to
time, is enabling Wake-on-LAN on an interface, from time to time.
This isn't a pressing issue, from my perspective, for core, at this
time.

>>
>>>     + Y: ftp
>>
>> If we really want an ftp client (I don't think we do), I'd rather have
>> lftp.
>
> I don't have strong feelings about what ftp clients rock nowadays. If
> lftp is the current best tool everyone uses, lets go for it! Will add
> N: for this one and add a Y: lftp line for now.

Surely curl and w3m can handle FTP sufficiently, right?  I can't see
where ftp is necessary, even for comfy.

>>
>>>     + Y: fuse
>>>     + Y: ntfs-3g
>>
>> If we want to support such file systems, then these should be !!
>
> You could be right; do we need those for runtime mounting such FS's or
> just for admin operations? Think that would give us guidance what to
> do here.

I'm +1 on these for !!

>>
>>>     + Y: gdisk
>>
>> Why? Repartitioning snappy sounds rather dangerous..
>>
>
> I assumed an admin likes a tool to manage additional harddisks and
> partition that are not part of the standard snappy partitions (e.g.
> boot, system-a, system-b, writable). You feel this tool would be
> useful for such cases?

Yeah, I think this would be good for comfy.

>>>     + Y: git-man
>>>     + ?: groff-base - anyone opinions?
>>
>> If you want to ship manpages, I think we need that.

Alternatively, all Ubuntu manpages are currently readable in HTML on
manpages.ubuntu.com, and every single manpage .gz file is also hosted
there, in a very predictable directory structure.

Moreover, there is a very small, 64-line shell script called dman(1)
that fetches the .gz on demand into a temporary file, and then pipes
it into the manpage viewer.  This could be a simple compromise, that
keeps our Snappy and Comfy images minimal, but still ensures that an
(Internet connected) admin trying to read some documentation can get
to it.  Thoughts?

See:  http://manpg.es/dman

>>>     + N: grub-legacy-ec2 - lets not do legacy in snappy world
>>
>> +1 :-)
>>
>>>     + Y: hdparm
>>
>> !!
>
> is this something we expect the majority of snappy admins to benefit from?
>
>>
>>>     + !!?: ntpdate - does this work in snappy, maybe in core?
>>
>> +1 for !!
>
> ack
>
>>
>>>     + Y: open-vm-tools
>>
>> Do we really want to support this? ATM snappy is mostly used in QEMU,
>> where you can't (or shouldn't) really start sub-VMs in.
>
> If this is not admin convenience, we should deal elsewhere with
> whatever use case caused this to into the cloud image. Anyone else has
> input?

These are needed, if the Comfy or Snappy image is being used as a
guest virtual machine inside of a VMware environment.  From the
package description:

Description-en: Open VMware Tools for virtual machines hosted on VMware (CLI)
 The Open Virtual Machine Tools (open-vm-tools) project is an open source
 implementation of VMware Tools. It is a suite of virtualization utilities and
 drivers to improve the functionality, user experience and administration of
 VMware virtual machines.

>>
>>>     + Y: os-prober
>>
>> Supporting multiple OSes in a snappy instance starts to sound really
>> complex and not like something which we want to support?
>
> indeed, oversight; changing to N
>
>>
>>>     + Y: patch
>>
>> N IMHO.
>
> indeed :). changed it to N, but happy to hear different opinions.
>
>>
>>>     + Y: perl
>>>     + Y: perl-modules
>>
>> This should be D. Or !! if we want to support Perl in snappy OS.
>
> ack, for sure D, done.
>
>>
>>>     + ?: policykit-1
>>
>> D
>>
>>>     + Y: powermgmt-base
>>
>> D please. This stuff is oooooooold and crufty.
>>
>>>     + Y: rsyslog
>>
>> Or just enable persistant journal?
>
> Maybe open a separate thread to discuss our general logging approach?
> Will make this ? to remind us to think about it.

Hmm, yeah, let's discuss more.  I could certainly see, in a Comfy
environment, wanting to rsyslog a *bunch* of devices I'm testing to a
single rsyslog server.

>>
>>>     + Y: sgml-base
>>
>> D
>
> ack
>
>>
>>>     + ?: ureadahead - on first glance feels unlikely needed if we dont
>>> need in the core system itself
>>
>> Not necessary, doesn't work under systemd anyway.
>
> ... and gone :)
>
>>
>>>     + ?: xauth - not clear why xauth is needed on cli comfy systems
>>
>> It would be useful to install and run an X app in snappy with X
>> forwarding (e. g. ssh -X).
>
> OK, guess we could bring this back - potentially elsewhere - once we
> solve this scenario (in case we do) :). marking as N for now.
>
>
>>
>>>     + ?: xdg-user-dirs - feels not relevant for snappy world with its
>>> own set of dirs; maybe snappy-user-dirs in core? :)
>>
>> +1, snappy pretty much ignores FHS and XDG dirs anyway.
>
> OK, i take that as a N. done!

Oh man.  You want to piss off some XDG nerds?  Show them the Snappy
filesystem layout :-)  And then duck!

> Thanks a lot Martin for going through this long list and giving guidance!
>
>>
>> Martin
>> --
>> Martin Pitt                        | http://www.piware.de
>> Ubuntu Developer (www.ubuntu.com)  | Debian Developer  (www.debian.org)
>>
>> --
>> snappy-devel mailing list
>> snappy-devel at lists.ubuntu.com
>> Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/snappy-devel
>
> --
> snappy-devel mailing list
> snappy-devel at lists.ubuntu.com
> Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/snappy-devel
>