ZFS feature flags

[Richard Laager]

On 10/19/19 4:43 AM, Colin Watson wrote:
> I don't have much to say about most of this, but noticed this bit:
> 
> On Thu, Oct 17, 2019 at 09:48:42PM -0500, Richard Laager wrote:
>> The same implementation could (and if I have my way, will) be used to
>> provide a features=grub mask. This would be used for the boot pool
>> (bpool) to limit it to the features supported by GRUB. This would avoid
>> the dangerous message in "zpool status" which tells you to run "zpool
>> upgrade" on your bpool which would then break booting from it.
> 
> Isn't that a dubious and confusing way to spell it?

I'm certainly open to a different name.

> After all, like any
> other ZFS implementation, GRUB's ZFS implementation has gained features
> over time, and it wouldn't surprise me if it continued to do so.  It
> sounds like you'd need a set of versioned features for this as well as
> for features=portable; but it's not clear how the decision of when to
> promote a feature set to the unversioned level would be made,
> particularly given GRUB's rather slow and unpredictable release cycle
> and the widespread practice of backporting features by distribution
> maintainers.

The proposal, as I understand it, is that the unversioned
features=portable is defined as the set of features common to all the
latest releases of specific "tier 1" ZFS platforms: FreeBSD,
illumos-gate*, Ubuntu LTS, and ZoL**. A feature is promoted into the
unversioned features=portable when it is available in all of those
platforms' latest releases. As new platform releases push out older
releases, the set of common features expands and the definition of
features=portable can be updated.

* illumos-gate, which I think doesn't have releases, was going to use a
time-based cutoff.

** In practice, unless Ubuntu backports something from git master or
writes their own ZFS feature, Ubuntu LTS will always be a subset of the
latest ZoL release. Thus, Ubuntu LTS will be the limiting factor for the
common set, not the latest ZoL release.

The versioned features=portable-YYYY are simply features=portable frozen
in time on January 1 of YYYY. The versioned ones are not driving
anything design-wise, so they are mostly irrelevant for this discussion.
If people also want versioned grub-YYYY, that's trivial to do.

I think there are two approaches to take for features=grub. Originally,
I was thinking that features=grub would be the set of features that are
supported by GRUB on the current system. This solves the `zpool status`
/ `zpool upgrade` issues for the boot pool, which is the main goal.

However, that definition could hamper dual-booting scenarios, so
features=grub should probably be similar to features=portable. It should
be the set of features common to all the latest releases of specific
"tier 1" ZFS platforms that use GRUB (specifically GRUB 2, not GRUB
Legacy) as their bootloader. I don't think that FreeBSD uses GRUB, and
neither illumos nor ZoL are whole distros, so in practice this is just
Ubuntu LTS at the moment. It may be desirable to expand this to include
other distros where root-on-ZFS is a thing, like Arch and Gentoo. In
other words, features=grub should be the features supported by GRUB in
the latest release of whichever distros people might realistically want
to dual-boot off the bpool.

Whether the feature arrives in Ubuntu 20.04's GRUB by GRUB release or
distro backport is irrelevant. The only thing that matters is which
features are supported.

An additional subtlety is that GRUB opens the pool read-only, so every
read-only feature is "supported" from a GRUB perspective. So what GRUB
"supports" on, for example, Ubuntu 20.04 is the union of the
features-for-write actually supported by GRUB in Ubuntu 20.04, plus all
read-only compatible features supported by the kernel ZFS implementation
on Ubuntu 20.04. (Dependencies have to kept in mind here. If feature X
is read-only compatible but depends on feature Y which is not read-only
compatible, and feature Y is not supported by GRUB, then feature X is
not supported by GRUB.)

-- 
Richard