the fragile boot process [on encryption setups and home data]

[Xen]

Not sure if this list is getting read much but:



- in a default install (now Talking Kubuntu 15.10) everything works but 
a small thing has to go wrong for the system to fail entirely
- this is not a resilient thing and it should get changed if the system 
is to be anything that allows people to actually customize something.


Example, and point in case:

- the partitioner in the installer allows to create logical partitions, 
but not extended ones, which is silly.
- I have to go and open the command line prompt to use a command line 
partitioner to get the partitioning scheme I want
- this time I created a dual LVM setup with the 2nd LVM encrypted for 
data

So my scheme is now:

/dev/sda1: /boot
/dev/sda5: LVM
lvm #1: root (16 GB)
lvm #2: swap (6 GB)

/dev/sda6: LUKS
/dev/mapper/sda6_crypt: LVM
lvm #1: thinpool
lvm #2: storage volume

So of course this is something different and exerimental that I'm doing. 
Nevertheless, the system should not bug out for minor things.

* If you have a swap configured in fstab, but there is something wrong 
with the swap, the boot process will not complete, even though swap is 
not necessary for booting the system
* If you have any mount configured in fstab, but it unavailable: the 
same thing happens.

I created two different LVM so that one can be encrypted and the other 
one doesn't have to be, for my current use case scenario.

The benefit, of course, SHOULD be that the system can boot without 
unlocking the LUKS container. However, there are two problems with this:

1. if you don't open the LUKS at boot, the LVM is unavailable; if the 
LVM is unavailable, the mount will fail. Perhaps I can solve this by 
specifying noauto in the fstab options. Otherwise, it will simply hang 
the system. No boot. No indication why either (except that I know of 
course). Error logs: unusable, unhelpful, worthless.

2. if you don't open the LUKS at boot, there is no user interface for 
the GUI to allow you to open it prior to logging in with some user, and 
even when you are logged in, I am not aware of any good interface to 
manage LUKS devices, although hopefully it will open it when I double 
click the device.

The point in case is that I want my /home on the encrypted device.

a. /home is not or should not be required prior to logging in
b. If you did log in (e.g. using a /home located on the root filesystem) 
and you opened the crypt device after, you'd have a problem and you'd 
have to log out, and log back in, just to (hopefully) get the new 
overlayed /home from your crypt device
c. In principle your home should be distinct from the root filesystem, 
because both may not necessarily have the same secrecy demands.
d. The current Linux system only supports two modes: encrypt your user 
homes with eCryptFS, or encrypt the entire volume/block device by 
mounting it AT boot -- there is nothing in between.

e. It is hence not possible to really have non-boot-mounted crypt 
devices unless they are really not used for anything during the normal 
system startup, especially when it comes to the /home structure.

f. Philosophically and principally, if /home contains your user data, it 
is not required for system functioning before logging into any user, 
nevertheless, the system will not work without it.
g. This makes it scarcely possible to locate /home on any other volume 
in a real sense, such as removable devices. Not that this should 
necessarily be possible, but I do think the ability to /boot/ your 
system without /home being available in the very beginning to be 
important. Simply because unlocking your /DATA/ structures is really 
something that ties in well with logging into a system, instead of 
something that ties in well with booting the system.

I think a real good encryption setup requires these 2 componenents:
- differentiating between system encryption and data encryption
- the ability to choose to unlock at various stages.

Traditionally in Windows of course this was also not really possible, 
but people used e.g. TrueCrypt to unlock additional volumes after 
logging in. User files in Windows also need to be accessible at boot, or 
at logon, which means there are always some user files on the system 
volume, at least if there is not any kind of network roaming setup in 
the computer/network system. This means on Windows, people have always 
tried to (or at least, I have) put their big and important and 
meaningful user files on some other volume, while accepting that and 
living with the fact that some files would always be created on that 
system volume. It is possible to relocate your C:\Users I believe, but 
not sure what happens then. You can also relocate your "libraries", 
which is more common I think.

Nevertheless, the fact that /user/ data is on a /system/ volume is 
strange to begin with.

And if it does not need to be on that /system/ volume, then why require 
it for booting, right?

I also firmly believe this:
- /home is not the best place to store bigger data collections.

So what I have done today, as a new experiment, once more is to:

- create one "/store" volume
- create subdirectories beneath that for ./media, ./backup, and also in 
this case ./home.

- personally I don't like eCryptFS even if perhaps it is the only way to 
encrypt user homes individually.

- my computer has always been my personal device; even if it wasn't, I 
would not like to use encrypted eCryptFS homes, but I would rather seek 
to create containers or volumes, in a certain sense block devices, that 
would enable additional volumes for me that I can decrypt (open) at 
will.

That model that I therefore seek to employ is more along these lines:

- seek to determine whether or not to encrypt the base system 
(non-personal, non-individual files, system files, system logs, etc.)
- seek to determine whether or not to encrypt data structures that may 
coincide with partitions, devices, etc. We are talking about external 
harddrives, usb-sticks, and all of that kind; but also partitions or 
volumes on the local disk that have been reserved for semi-personal 
data.

- choose whether you want (all) user homes encrypted or not. If you do, 
every user may need to know the password for that. But the benefit is 
that the system can boot without unlocking it. Another benefit is that 
you can put "general data" on the same volume as "personal data" (that 
is really the reason for it).

Ideally, encryption can perhaps fall into 3 stages:
1. complete system encryption (unlock required to boot the system, no 
information leaks)
2. unlocking data files of a general kind (often located on external 
drives and such)
3. unlocking data files of a specific, personal kind (can be in /home, 
on dedicated volumes, USB-sticks, etc.)

On both Windows and Linux I have never wanted to put my most important 
stuff in /home. The prime reason is that if I put it somewhere else, it 
is more mobile, more resilient. In Windows, you can put data like this 
in its own drive letter such as E:, and this makes it more easily 
accessible. In Linux, /home typically belongs to the root filesystem. If 
it doesn't, it might take up a lot of space you don't need.

Who stores some big video or audio collection in /home? That's no place 
for it. Neither on Windows, nor on Linux.

That's why we have these two phenomona:
1. big data collections like sitting on their own disk/partition/device
2. user home data is for all users, but where does it belong? It doesn't 
need a lot of space, but it is not really system data.

User accounts belong to a system, or can be sourced from some network 
roaming user thing.

In general on a local system I would recommend /home to be its own 
volume, BUT.

* Linux/unix groups are not really used much in present day Linux 
systems
* There is not really a real shared concept of "shared" data on our 
systems
* It requires a lot of plumbing to set anything like that up.

* If you give too much space to /home, it is often a waste, but too 
little and you don't have enough temp space to do stuff.
* I prefer a btrfs-agnostic approach, more liking to use LVM for that
* Grub support is not there for thin LVM
* We cannot have thinly provisioned LVM on our root or /boot systems, in 
a regular system
* One volume for system and one volume for data is the setup most people 
have used in Windows (that knew what they were doing).
* A system drive that can be encrypted separately from data is a boon.
* You still can encrypt everything in one go (one big encrypted LVM) if 
you want that but not very powerful.
* Encrypting subvolumes individually is a bit troublesome in Linux due 
to lacking GUI for it.
* Having a separate data volume allows you to decrypt it at boot or at 
logon, very pleasant if you can do it after logging into your user.
* What to do with /home? It is a strange beast.

* Personally I favour at this point a GUI element that allows unlocking 
a dedicated /home or some volume that /home resides on, prior to logging 
on, which would then reload the user list by scanning /home, thereby 
offering the /home that someone just unlocked.

* Using your user password for unlocking your files (eCryptFS) is not a 
safe thing from my perspective (anyone that you give your user password 
also has your files, all of them).
* I have never had other users on my computer but in a bigger 
home/network environment, roaming profiles seems to be the way anyway, 
or at least a way to sync your files to a network location.

* Different computers with different (temporary) user created files, is 
terribly annoying.
* In the linux world often times these user files were then mounted from 
some NFS share.
* The idea of NFS is that the system mounting the share needs to present 
the 'credential' or authority for the entire share, not for individual 
profiles.
* Having access to the system then implied having access to the share as 
well. You could either encrypt the system, or the entire remote home 
directory source volume (on that host) but if you wanted encryption on 
the remote side, you would need to encrypt that entire thing and unlock 
it at boot (for example).
* You cannot easily have individual block level encryption without 
wasting a lot of "free" space. Every block file uses the maximum space 
allotted to that user. Hence, eCryptFS.

* There is also the issue of system-specific individual files (like 
configuration files) and non-system specific ones. There has not really 
been a good separation of the two, which means I guess that people have 
been putting their real data files away from /home.

Should /home only be for configuration data and not for actual user 
created files?
Should we keep stuck with this mash up, this intermingling of 
system-specific and user-specific files?
Has there ever been a really good model that showed what's what? In 
Windows, one type might be stored on "Local" while another might get 
stored on "Roaming". In general for instance big lumps of data (ie. 
browser caches) are stored in the "Local" while real meaningful data is 
stored in "Roaming", hence, this data is not moved across the network 
(the big cache data).

I think anyone will attest that what is really important for them, the 
stuff they'd put on a personal usb stick, or a personal Dropbox account 
(or whatever you'd use, OneDrive, etc) - that stuff stays the same from 
system to system and has nothing to do with the current computer they 
are using. Then, it might also require different forms of encryption or 
would want to sit on some other locations.

Linux has a lot of dot files that pertain to system configuration (for 
that user, most notably the desktop environment) and which may also 
contain caches. Browser cache may be sensitive, but less sensitive than 
e.g. email cache (think Thunderbird).

- I don't think in Linux there is this possibility of differentiating 
between one type and the other.

- typically ALL data would need to be either roaming (sourced from a 
network share) or local (on the current computer).

- one can use overlay filesystems to combine two /home structures (such 
as aufs) but this gets rather tricky and dangerous and complicated 
pretty quick.

- any solution that works for a local system should work for a system 
that has roaming profiles.

- a well thought-out overlay or inclusion system might be able to flag 
individual files/directories according to what is needed for them or 
where they are coming from; you could have your /home populated with 
files from various sources, but this is not what we have available 
today.

- because of browser caches and all of that, making backups of /home 
typically means your /home/user is going to be fairly large.

You may not even want that, that begs the question again: where will you 
store your REAL user data?

For many people currently the solution will be to create some data 
volume and then link their libraries to that location. Then the data 
volume can be network source as well. But still what is missing is the 
ability the unlock such a volume prior to logging in.

What are your thoughts on this?


DO YOU think we will need to have a "unlock vault" functionality 
pre-login in our various environments? (Unity, KDE, etc.)
DO YOU think we need a better default solution for system (root) versus 
local (non-root) alterations to a system?
IS THERE a place for /usr/local still, and, conversely, should THAT be 
our data volume?
WOULD WE be better off having something like /store or /data as an 
accessor to our data volume?
DON'T we have a bit too many root subfolders like /opt and /srv and /sys 
and /run and /proc these days that unneccesarily clutter the filesystem?
DOES linux need better support for installing programs as a non-root 
user? Ie. the thing that /usr/local is supposed to be for?

do you feel the system should not hang during boot because of flawed 
fstab entries?
do we need a better gui for mounting individual entries, encrypted or 
not?
do you consider the current LUKS system proficient enough, or do we need 
more layering (ie. the thing I proposed) (which is having system 
encryption AND user data volume encryption (and not just eCryptFS)).

do you feel like me that the leaking of information at the boot 
decryption prompt, is a bad thing?
should an encrypted system ever have more than one volume that needs to 
be decrypted at boot (by way of a password)
would we not be better off having just one type of system encryption (in 
this sense) never requiring more than one password?
is unlocking additional volumes not simply better done post-logon or 
pre-logon? don't we need a gui for that?

(is it not annoying that on Windows on an international keyboard, 't 
just writes 't, but on Linux it writes nothing at all? :P)

is the default setup of eCryptFS sufficient? I don't think so. I don't 
even think the default "encrypted LVM for the whole drive" is 
sufficient. Because if you do that, you cannot have a differently 
encrypted data volume (because you'd be layering two encryptions on top 
of each other, which is not really nice). There are at least 3 or 4 
different volumes up for consideration:

- root
- home
- big user data
- boot

and also:
- swap

Especially "big user data" but also /home, in my case, is up for not 
being on the same LVM as root/boot/swap. Simply because you'd rather 
encrypt the entire LVM rather than individual volumes. Who has the 
answers here? I don't know, if anyone does.

Regards,

Bart.