Installing Xenial with root-on-ZFS on top of LUKS, UEFI: no cryptsetup in grub

[Regulus Kean]

Hello,

it was suggested to me on IRC that I post my question to these two lists.
I hope that this is alright, and that you could give me some guidance.

I'm trying to install (on a Lenovo T460s laptop) 16.04 Xenial, in a way
that is cryptographically as secure as I manage.
I learned that it is no longer necessary to have /boot unencrypted, and
wanted to try having it encrypted as well. Had I managed to make it work,
I also have proceeded with signing the bootloader and using UEFI secure
boot.
Also, I wanted to use ZFS for the entire disk (except EFI System Partition).

I couldn't find a text outlining what to do to achieve this, but various
guides showing how to do subsets of what I had in mind [1-3].

I'll post the exact steps I took below, but at the first attempt to boot
into the newly installed system, grub2 asks for a password (expected, to
get access to /boot if I understood correctly), but then drops into
busybox without asking for the password again (this I did not expect,
since I thought the kernel booting would try to decrypt the LUKS volume
again).

>From what is shown on the console, I can see no signs of an attempt to
decrypt the LUKS container. I do have /conf/conf.d/cryptroot, however with
contents "target=sda2_crypt,source=UUID=<UUID of
/dev/sda2>,key=none,discard". /dev/mapper is empty, except for "control",
and "zpool import rpool" fails of course.

At the busybox prompt, I saw that cryptsetup is not accessible, so I
couldn't try to do a luksOpen myself. Also, I have to admit I failed to
get cryptsetup into the boot image.

I realize of course that this is a peculiar setup, but I figure that there
may be something to be learned here, and maybe bugs to be found. I have
come across a few known ones already doing this.
I also experimented a short amount of time with replacing ZFS with XFS,
but that didn't do much help. If it removes complexity, I'd be very happy
to go with any other filesystem, of course.

Any kind of input is very much appreciated! I am of course willing to put
work into this, but sadly don't know how to proceed from here.

Thanks!

Link list:

[1]
https://askubuntu.com/questions/729673/ubuntu-full-disk-encryption-with-encrypted-boot
[2]
http://www.thecrosseroads.net/2016/02/booting-a-zfs-root-via-uefi-on-debian/
[3] https://github.com/zfsonlinux/zfs/wiki/Ubuntu-16.04-Root-on-ZFS

Commands to reproduce, executed from a Xubuntu 16.04 amd64 installer disc:

apt-get update
apt-get upgrade -y

gparted /dev/sda #create GPT partition table, a 200MB partition for the
EFI System Partition (sda1), remainder for LUKS (sda2)
cryptsetup -v --cipher aes-xts-plain64 --key-size 512 --hash sha512
--iter-time 5000 --use-random luksFormat /dev/sda2
cryptsetup luksOpen /dev/sda2 sda2_crypt

apt-get install -y zfsutils-linux
zpool create -o altroot=/mnt -m none rpool /dev/mapper/sda2_crypt
zfs set atime=off rpool
zfs set relatime=on rpool
zfs create -o mountpoint=none rpool/ROOT
zfs create -o mountpoint=/ rpool/ROOT/ubuntu-16-04-1
zfs create -o mountpoint=/home rpool/home
zfs set mountpoint=/rpool rpool
zpool set bootfs=rpool/ROOT/ubuntu-16-04-1 rpool
zpool export rpool
zpool import -R /mnt rpool
mkdir -p /mnt/etc/zfs
zpool set cachefile=/mnt/etc/zfs/zpool.cache rpool

apt-get install -y debootstrap
mkdir -p /mnt/var/tmp
chmod 1777 /mnt/var/tmp
debootstrap xenial /mnt
echo T460s > /mnt/etc/hostname
echo '127.0.0.1 T460s' >> /mnt/etc/hosts
echo 'auto lo' > /mnt/etc/network/interfaces.d/lo
echo 'iface lo inet loopback' >> /mnt/etc/network/interfaces.d/lo

for f in dev dev/pts proc sys ; do mount -v --bind {,/mnt}/$f ; done
chroot /mnt /bin/bash --login

echo 'deb http://archive.ubuntu.com/ubuntu xenial main universe' >
/etc/apt/sources.list
echo 'deb http://security.ubuntu.com/ubuntu xenial-security main universe'
>> /etc/apt/sources.list
echo 'deb http://archive.ubuntu.com/ubuntu xenial-updates main universe'
>> /etc/apt/sources.list

sed -i -Ee 's/# (en_US.UTF+)/\1/' /etc/locale.gen
locale-gen

apt-get install -y man vim

passwd #set root password

apt-get update
apt-get install -y zfsutils-linux zfs-initramfs linux-image-generic #when
GRUB asks for it, I select *no* device to install it on

apt-get install -y dosfstools
mkdosfs -F 32 -n EFI /dev/sda1
mkdir /boot/efi
echo PARTUUID=$(blkid -s PARTUUID -o value /dev/sda1) /boot/efi vfat
defaults 0 1 >> /etc/fstab
mount /boot/efi

apt-get install -y grub-efi-amd64
export ZPOOL_VDEV_NAME_PATH=YES #grub-probe fails if this is not done: see
https://bugs.launchpad.net/ubuntu/+source/grub2/+bug/1527727/comments/29
apt-get autoremove -y #remove grub-pc-bin
echo 'GRUB_ENABLE_CRYPTODISK=y' >> /etc/default/grub #otherwise
grub-mkconfig complains ("=y" is the only correct choice, even though
grub-mkconfig suggests "=1" instead)
echo sda2_crypt UUID=$(blkid -s UUID -o value /dev/mapper/sda2_crypt) none
luks,discard >> /etc/fstab
grub-install --target=x86_64-efi --efi-directory /boot/efi
--bootloader=ubuntu --boot-directory=/boot/efi/EFI/ubuntu --recheck

#grub looks for sda2_crypt in /dev for some reason:
ln -s /dev/mapper/sda2_crypt /dev/
echo 'ENV{DM_NAME}=="sda2_crypt", SYMLINK+="sda2_crypt"' >
/etc/udev/rules.d/99-local.rules

echo sda2_crypt UUID=$(blkid -s UUID -o value /dev/sda2) none luks,discard
>> /etc/crypttab

sed -i '139 s/$/ || true/' /usr/sbin/grub-mkconfig
echo 'GRUB_DEVICE="/dev/mapper/sda2_crypt"' >> /etc/default/grub
echo 'export CRYPTSETUP=y' >
/usr/share/initramfs-tools/conf-hooks.d/forcecryptsetup
echo target=sda2_crypt,source=UUID=$(blkid -s UUID -o value
/dev/sda2),key=none,discard > /etc/initramfs-tools/conf.d/cryptroot

sed -i '13 s/^/#/' /etc/default/grub
echo 'GRUB_PRELOAD_MODULES="luks cryptodisk"' >> /etc/default/grub
echo -n 'GRUB_CMDLINE_LINUX="cryptdevice=' >> /etc/default/grub
echo -n $(blkid -s UUID -o value /dev/sda2) >> /etc/default/grub
echo ':sda2_crypt crypto=sha512:aes-xts-plain64:512:0:"' >> /etc/default/grub

grub-mkconfig -o /boot/efi/EFI/ubuntu/grub/grub.cfg
update-initramfs -c -k all