Isolated Base Installation?

Wed Nov 1 00:36:33 UTC 2006

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

In the face of new technologies I'm thinking about pulling some ideas
out of Phasic Linux, an old Linux distribution idea I had when I was
younger, and throwing them at the Ubuntu community to see if they come
up with anything.  The first of these is the Isolated Base Installation.

Basic information comes from my old files, which I've publicized on the
machine behind me.  These are old and ugly.  New ideas are presented
here which go beyond the original ideas, but are in a crude and
unrefined brainstorm state.

  http://bluefox.kicks-ass.org/stuff/bluefox/plx/www/admin.html

NEW TECHNOLOGIES MAKING THINGS POSSIBLE

I would like to first identify the new technologies that will make the
things described here possible.

 - FUSE Filters.  FUSE will have a FilterFS API released in the future,
   which will allow a FUSE File System to read through itself.  In other
   words, if you mount EncFS on /home, and then mount CompressFS on
   /home, you will write to CompressFS, which will compress and write to
   EncFS, which will encrypt and write to the underlying disk.

http://thread.gmane.org/gmane.comp.file-systems.fuse.devel/3595/focus=3626

 - UnionFS.  FUSE filters can be used to implement a FUSE UnionFS
   transparently.  This UnionFS could be designed to union multiple file
   systems and hide the underlying mount point, for example /union_base
   could be hidden if the UnionFS mounted over /.

 - Snapshots.  FUSE file systems can be used to make snapshots.  A
   snapshopt file system exists IIRC, which records changes made and
   shows the unchanged view in another mountpoint.  This functionality
   is not necessarily important (a snapshot will look like the system
   had a failure at runtime) and I will only explain why I don't like it
   here.

THE IDEA

In the original docs for Phasic Linux (as recent as January, 2005; how
shameful!), I describe something called Isolated Base Installation, or
IBI.  Effectively, IBI is a method for preserving the install time
settings such that a 'reinstall' is facilitated with a call to 'mkfs'.

The basic premise for IBI is that when the system is installed, the root
is made read-only and kept that way.  Changes are redirected to another
file system.  If you break it, format that other file system.

Further ideas discussed here include compressed root; snapshots; and
"live review."

BASIC IBI

In most basic implementation, IBI would be implemented by using a
read-only root partition.  The root would necessarily be isolated from
both /home and /boot; the file system itself is read-only, in the event
of a crash from power drop and file system corruption it needs to be
recoverable.

The root would be mounted and have a read-write root changes partition
mounted on top of it using a UnionFS FUSE filter.  From here we would be
able to begin booting; / is complete, and can mount /boot and /home as
needed.

  [NOTE 01:  We already can see the unclear use of /var makes this a
   poor choice for database, Web, and Squid cache servers.  If we
   isolate /var, then the snapshot doesn't get the dpkg database et al;
   whereas if we include it, we back up multi-gigabyte databases or Web
   sites  where we only intend to provide a functional SYSTEM which may
   fail without harming the database.]

If the system failed, the user would boot into "Maintenance Mode."
Given a menu, he would select "Restore System."  The contents of the
read-write target for root would be destroyed, and the system would
continue booting as normal (no reboot required).

  [NOTE 02: Information about where real / and the read-write target sit
   is probably in /boot]

IMAGE-BASED IBI

Basic IBI is an annoyance because you have a partition for read-only;
another partition for read-write; and then isolated /home and /boot.
With FUSE filters, a superior UnionFS can be made that relies on images.
 Isolated /home and /boot are no longer necessary.

The prerequisite for this is a UnionFS that reads underneath itself and
allows mounting on the bottom.  This means a UnionFS has to be able to
mount over /; read the physical files on disk; and mount a union such
that files not present on the physical file system underneath it would
fault to the union.  With this, we're ready for image-based IBI.

 - For this discussion, assume images are in /.root_images/
 - Mount points are created in /.root_images/mnt/

The process for booting an image-based IBI system would start by
mounting the real root file system read-write.  Then the UnionFS would
be brought up overtop of it.

The next step is to read the /etc/ibi/roottab file, which specifies the
order in which to mount root images.  The first image specified would
be, for example, /.root_images/root mounted to /.root_images/mnt/root.

The UnionFS is set up to union each image mounted in order from bottom
up.  That is, /.root_images/mnt/root/ would be on the bottom; then
possibly /.root_images/mnt/snapshot_1/.  The reads would proceed
top-down:  if a file is not found (or whited out) on /, we read
/.root_image/mnt/snapshot_1/; and if not there, then /.root_image/mnt/root/.

This has two major impacts:

 - Any file being created or modified is written to the physical disk /
 - Any file being read that is not on the physical disk / is read down
   through the images mounted

Because of this, restoring the system can be done simply by wiping /
excluding /.root_images/ and /home/.  Any snapshots taken would include
/etc (where roottab lies) and /boot (where kernels lie).

It would be possible to make two types of snapshots from this.  The
"Full" snapshot would read through the entire union, omitting /home and
/.root_images/.  An "Incremental" snapshot would pack up only what was
on the physical / file system except for /home/ and /.root_images/.
Both are potentially useful; "Full" can be for any post-install or
post-upgrade while "Incremental" can be used for customization after
base install or for backing up before dangerous system modifications.

Again, if the system failed, the user would boot into "Maintenance
Mode."  Given a menu, he would select "Restore System."  The contents of
the read-write target for root would be destroyed, and the system would
continue booting as normal (no reboot required).  The contents of /boot
would have to be copied back to / to keep kernels and grub.conf accessible.

Further, the user could create an image in Maintenance Mode by selecting
"Create Full Restore Point" or "Create Incremental Restore Point."  The
restore point would be created from the contents of the real / file
system, and the contents of that file system would be wiped except for
/home and /.root_images/.

Problems with this approach include that / cannot simply be wiped, as it
contains both the back-up images and /home.  Utilizing a separate
partition as the union target is an option, if you can have the UnionFS
exclude /.root_images/ and /home from the union and redirect all other
writes.

COMPRESSED IMAGE IBI

Compressed image IBI is dead simple.  Image based IBI, but use SquashFS
for the image.  There you go, now the images are compressed.

SNAPSHOTS

Image based IBI defines a Maintenance Mode that allows for creating Full
or Incremental Restore Points.  These are both snapshots.  In the case
of Incremental, you can roll back system changes easily by taking
snapshots before making them.

Note that producing a method for taking a snapshot during shutdown, just
before unmounting file systems, would provide a short circuit for
generating LiveCDs with persistent root.  The same code used to do
everything else here could be used to union a LiveCD to an Incremental
image stored on a USB device.

LIVE REVIEW

Because we can create multiple Full and Incremental Snapshots, it is
logical that we should be able to boot any system state we have a
snapshot for.  This thinking is interesting, but any unioning with the
read-write root normally used would create a corrupt system.  We could,
however, treat this like a LiveCD and reap some benefits.

"Live Review" of any snapshot would be possible by replacing the
read-write root with a tmpfs.  The /home directory would have to
continue to be visible; but all other contents of the system would have
to be removed from play.  The images would be mounted as normal up to
the point desired; a tmpfs would be mounted overtop; and /home overtop that.

The /.root_images/ directory is probably not needed because it's
unlikely we'd want to take a snapshot of this; besides, we can't reboot
without losing the contents of /tmpfs.  We thus wouldn't make it visible
in our scheme, at least not intentionally.  If it happens to be easier
for it to be there (i.e. it takes work to hide it), we really lack caring.

- --
    We will enslave their women, eat their children and rape their
    cattle!
                  -- Bosc, Evil alien overlord from the fifth dimension
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.3 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iQIVAwUBRUfsDws1xW0HCTEFAQLX4g/+OlanCCzDMwpcIl+Q7eJFm6sXwMyn3mf6
ZKawdiadqtNUi+ae/Zo+Tfw5jvEKPl/N6RkZUMx0GAVZ+kpCqbDNu6xer4kq0s8n
ON1oetUmDbrRGPiL4fO0cPLvpAvxqQVbqZnUgXjmJPMyjf2Q1mIgKyJePw8mymQ4
n40PN1WFqGZaGS1g1GplLXWcZr0oQCgUMVysqwCOPpfKq77K+0fCXpHTrjSFtZTH
sfYVxJfbxAzMcigtwr/5T8Kq7EqNyydsgCrOxUjE4LwOwG6SIXTPdPIkwmbZj/82
ujU/pv1DEa0Vz/k4vPXhwWDRm2tkiVBBNVLShRJd5B/wwyUuo6qwUgMgFkWwSgky
Z9s2soz7LkALhJYtpOjN/FphkuozMZEEWdAKXJg1GniMb+cKKQfq/nq4q8akG9t3
6Jk2BQGbXQfRZNHYhvMzDfmjrjyMYpFUDz+HLJ0D+ThxGy15EKRRIGRza+Pgo9pB
78iwz1o4JVg7XpgRPa/zCOrWh69LERDqNwls79GzbhbsXEVXhq2xp5nl5eWxKPWA
Q8Hv0iiRL+6f67RhbB5CHvBKuG1PQhl2c1twJesZQePGAOFPrBsnlyFNeTh3bqPy
w4KM4keSgTwmzpoGc7ohicWnCBVsBChfzDr5VpTaKO/GU0FysGUgzm6RBORIpDTN
m1/GMd6JQQI=
=bMr2
-----END PGP SIGNATURE-----