classic mode (v1)

[Martin Pitt]

Hey Gustavo,

Gustavo Niemeyer [2015-11-24 13:15 -0200]:
> > I actually promised myself to not point this out for the umpteenth
> > time, but it seems the conversation on the gdoc stopped.
> >
> Sorry for making you come back onto this issue again. There might be
> information we're lacking, so your participation is appreciated here.

Heh, I wasn't sure where the "input information" ends and getting on
other people's nerves begins :-)

> > I *strongly* recommend to not do this. We have dozens of bug reports
> > of packages misbehaving on overlayfs; a particularly nasty case are
> > packages that fail to upgrade/install on overlayfs, which is an use
> > case which is relevant in the classic env (but not on a live system).
> >
> Do we know which specific packages are broken and what causes these
> failures, so we can have a more clear idea of where the line between
> working and non-working is?

We don't have an exhaustive list of those as in general we don't
do/offer upgrades on the live system and don't configure system
installations with overlayfs (for this particular reason). But you can
easily find examples: #940797, #1015348, #932663

A lot of these were duplicated to the master bug #882147, but this is
certainly far from complete. I've also seen weird install/upgrade
failures from dpkg itself when it unpacks the new version and hits
corner cases when it tries to atomically switch to the new version of
a file.

There's other bugs like #1465998 which people have run into which got
fixed in the meantime; but nobody has tested/used it to do massive file
operations like a dist-upgrade.

> It also degrades performance, causes stat() to misbehave (as the
> > overlaying is leaked left and right into userspace, so e. g.
> > "mountpoint" fails), and after a few rounds of dist-upgrades you still
> > have the original old file system which is then just wasting disk
> > space.
> >
> 
> In which way does stat misbehave, and do you have an idea of how the
> upstream feels about those issues so we can have a more informed view on
> the case?

In particular, the st_rdev field is broken. Depending on whether a
file (or perhaps also its containing dir) are unmodified (i.  e. from
the underlay) or modified (in the overlay) it has different results.
This is where the implementation "shines through" to userspace. Most
recent overlayfs has gotten quite a bit more predictable in that
regard (the out-of-tree version had a zero for the major number, and
completely useless minor numbers).

st_rdev is commonly used to determine whether a file/dir is a
mountpoint. As overlayfs doesn't implement name_to_handle_at() or
/proc/self/fdinfo/ properly (which are preferred and faster),
comparing st_rdev is the next best thing, but with overlayfs every
modified file appears to be a mount point if you query it that way.

There might be other issues, but that's the one that I recently
stumbled upon.

That said, even in a world where all of this wasn't an issue, the
degraded performance (Mark mentioned the slow raspi, on platforms
like that it would suck even more) and the wasted disk space are IMHO
sufficient reasons by themselves to not use this for a long-lived and
evolving/upgrading environment.

> > 2) All system level changes (like additionally installed packages in
> > >    ubuntu-classic) are lost on each snap update.
> >
> > How would that be? As soon as you e. g. upgrade a deb, it's in the
> > overlay, and if you change the same file in the underlay, that change
> >
> 
> Both Michael and myself have stated in this thread we're not changing the
> underlay and the overlay independently.

OK. But if you change the underlay, i. e. provide a newer version of
the snap that provides it, then you have little other choice than to
completely empty the overlay, i. e. reset the classic env. From what I
heard about the requirements, this is not what we wanted, but instead
the classic env should independently live/get upgraded?

> Well, it's really a blend of the two, but perhaps more like a schroot. You
> are playing in an environment inside another one, with consequences for how
> systemd, devices, etc, behave. At the same time, we do want people to
> install editors, debuggers, tools, etc, but people do that in schroots as
> well.

schroots are nice for building packages (as the installation etc.
happens automatically), but they are not really a good tool for doing
development in -- you'd have to set them up manually each time you
want to do stuff. This might also include changes in /etc, configuring
PPAs, and what not.

Thanks,

Martin

-- 
Martin Pitt                        | http://www.piware.de
Ubuntu Developer (www.ubuntu.com)  | Debian Developer  (www.debian.org)