ZPool not mounting at boot

[Liam Proven]

I did a lot of flailing around on this one, and I _may_ have solved
it, in a slapshot way. Not sure yet.

I run Ubuntu Server 20.04 64-bit on a Raspberry Pi 4, as a home NAS
server. It has 4 x 2TB USB3 drives in a ZFS RAID on a USB 3 hub which
also powers the RasPi. No fans, passive heatsink/case, silent except
for disk accesses.

I built this in May, described here:
https://liam-on-linux.livejournal.com/71093.html

It's been notable that Focal on the RasPi 4 was not very stable at first.
• My ethernet connection works but traffic pauses for from seconds to minutes
• wifi worked better, after I wrote a netplan YAML, but would randomly
drop the IPv4 address
• I had a lot of CPU activity from multipathd
• I had a bunch of snaps mounted continuing GNOME stuff - on a
headless server with no GUI
• no ZFS binaries were included or in the repo -- it all had to be
built from source from scratch. First time took 45 min with all 4
cores pegged

I removed snapd, multipathd, some bit of the GNOME search backend I
forget. This got me a hundred meg or so of RAM back, ended dozens of
always-active threads, and took idle CPU usage down to about 4%.

It ran usefully, but I had to connect to the wifi IP for consistent
performance. Ditto for management with ssh. Over wifi, it can take
writes of about 1GB/min which is quite good.

I use Webmin to monitor it and netatalk so my Mac can back up to it
with Time Machine.

Over the months, gradually, with updates, things have improved.

Ethernet performance seems stable now: no freezes.
Wifi keeps its IPv4 address for weeks at a time.
Automatic unattended-updates run regularly and email me the results.

But recently I noticed that I could connect to my shares after a
reboot. On investigation, the zpool is not mounting. I can do it
manually with zpool import vol0 but I should not need this.

Last night, I did an update, then `shutdown -r now`. It did not come
back up. No IP leases, nothing.

I had to pull the plug. That's a first and not a good one.

I did some Googling and found this: a StackExchange answer that lists
a load of systemd services that should be autostarting:
https://unix.stackexchange.com/questions/338260/zfs-mounting-only-one-of-my-pools-on-boot

I did them all. Only 1 gave an error. Last reboot, it did mount my
RAID successfully, but this reboot took about 10min and was rather
nerve-wracking -- in that time, I could not ssh into the box.

Now it's running happily again though, and my Mac is preparing a Time
Machine backup for the first time in 36 hours.


-- 
Liam Proven – Profile: https://about.me/liamproven
Email: lproven at cix.co.uk – gMail/gTalk/gHangouts: lproven at gmail.com
Twitter/Facebook/LinkedIn/Flickr: lproven – Skype: liamproven
UK: +44 7939-087884 – ČR (+ WhatsApp/Telegram/Signal): +420 702 829 053