[Bug 1661629] Re: upgrade of kernel fails with mkinitramfs: failed to determine device for /

Wed Jul 4 10:45:08 UTC 2018

On Wed, Jul 04, 2018 at 01:21:35AM -0000, John Gallagher wrote:
> > I'm not particularly familiar with this part of initramfs-tools, but
> it seems to me that essentials like the loading of the virtio_pci module
> would be short-circuited by your patch.
> 
> Yes, you're right, thanks for pointing that out. If the driver isn't
> built-in to the kernel, it would be omitted. I think what we need to do
> here is get a list of the devices that make up the root pool, probably
> from the output of `zpool list -vPL`, and then call 'block_dev_mod_add'
> on each device. I'll work on updating the patch.

That sounds like it might be the right approach, but I'm not sure. I'm a
bit confused as to exactly why update-initramfs fails here due to the
way it is being called - see below.

> > I was just pointed to
> https://github.com/zfsonlinux/zfs/wiki/Ubuntu-18.04-Root-on-ZFS and told
> that ZFS filesystems aren't expected to be in /etc/fstab according to
> those instructions. Is this correct? I haven't checked but it seems that
> initramfs-tools wouldn't then fail under these conditions.
> 
> It's true that we don't have the filesystems listed in /etc/fstab. Can
> you elaborate on how you anticipate that would affect building the
> initramfs?

Ah - that won't be the issue then. I was suggesting that if you had the
filesystems in /etc/fstab, then that might be the cause of your problem.
It sounds like that isn't the case then.

> > Does this apply and work correctly on the current Ubuntu development release (Cosmic) please? We land changes there first and only once successful consider backporting to existing releases as appropriate.
> Is it possible to upgrade my machine to the development release? If not, I might be able to modify our build process to build an Cosmic image that uses zfs on root.

I suggest that you test using VMs, and initially use development release
images rather than upgrading up to it. But yes, you can upgrade using
do-release-upgrade(8) (in steps, with '-d' for the final step).

It looks like kdump-tools might be causing the issue here, rather than
this being (directly, at least) being a problem in initramfs-tools.
kdump-tools drops in a kernel postinst.d/ script. This script calls
mkinitramfs but with a modified MODULES=dep configuration option:
https://git.launchpad.net/ubuntu/+source/makedumpfile/tree/debian/kernel-postinst-generate-initrd?h=applied/ubuntu/devel

It might be easier to resolve this by considering this as "mkinitramfs
fails on ZFS root without /etc/fstab lines for the ZFS filesystems when
MODULES=dep is used". If that's accurate (I haven't tested).

Could you perhaps see if you can remove kdump-tools from the equation
like this?

-- 
You received this bug notification because you are a member of Ubuntu
Foundations Bugs, which is subscribed to initramfs-tools in Ubuntu.
https://bugs.launchpad.net/bugs/1661629

Title:
  upgrade of kernel fails with mkinitramfs: failed to determine device
  for /

Status in initramfs-tools package in Ubuntu:
  Confirmed

Bug description:
  When upgrading packages with "apt upgrade" on Ubuntu 16.04.1 LTS with
  root on ZFS I get this error:

  Setting up linux-image-4.4.0-59-generic (4.4.0-59.80) ...
  Running depmod.
  update-initramfs: deferring update (hook will be called later)
  Examining /etc/kernel/postinst.d.
  run-parts: executing /etc/kernel/postinst.d/apt-auto-removal 4.4.0-59-generic /boot/vmlinuz-4.4.0-59-generic
  run-parts: executing /etc/kernel/postinst.d/initramfs-tools 4.4.0-59-generic /boot/vmlinuz-4.4.0-59-generic
  update-initramfs: Generating /boot/initrd.img-4.4.0-59-generic
  run-parts: executing /etc/kernel/postinst.d/kdump-tools 4.4.0-59-generic /boot/vmlinuz-4.4.0-59-generic
  kdump-tools: Generating /var/lib/kdump/initrd.img-4.4.0-59-generic
  mkinitramfs: failed to determine device for /
  mkinitramfs: workaround is MODULES=most, check:
  grep -r MODULES /etc/initramfs-tools/

  Error please report bug on initramfs-tools
  Include the output of 'mount' and 'cat /proc/mounts'
  update-initramfs: failed for  with 1.
  run-parts: /etc/kernel/postinst.d/kdump-tools exited with return code 1
  Failed to process /etc/kernel/postinst.d at /var/lib/dpkg/info/linux-image-4.4.0-59-generic.postinst line 1052.
  dpkg: error processing package linux-image-4.4.0-59-generic (--configure):
   subprocess installed post-installation script returned error exit status 2

  version of initramfs-tools is 0.122ubuntu8.8

  Output of mount is:

  sysfs on /sys type sysfs (rw,nosuid,nodev,noexec,relatime)
  proc on /proc type proc (rw,nosuid,nodev,noexec,relatime)
  udev on /dev type devtmpfs (rw,nosuid,relatime,size=8066020k,nr_inodes=2016505,mode=755)
  devpts on /dev/pts type devpts (rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000)
  tmpfs on /run type tmpfs (rw,nosuid,noexec,relatime,size=1623576k,mode=755)
  rpool/ROOT/ubuntu on / type zfs (rw,relatime,xattr,noacl)
  securityfs on /sys/kernel/security type securityfs (rw,nosuid,nodev,noexec,relatime)
  tmpfs on /dev/shm type tmpfs (rw,nosuid,nodev)
  tmpfs on /run/lock type tmpfs (rw,nosuid,nodev,noexec,relatime,size=5120k)
  tmpfs on /sys/fs/cgroup type tmpfs (ro,nosuid,nodev,noexec,mode=755)
  cgroup on /sys/fs/cgroup/systemd type cgroup (rw,nosuid,nodev,noexec,relatime,xattr,release_agent=/lib/systemd/systemd-cgroups-agent,name=systemd)
  pstore on /sys/fs/pstore type pstore (rw,nosuid,nodev,noexec,relatime)
  cgroup on /sys/fs/cgroup/freezer type cgroup (rw,nosuid,nodev,noexec,relatime,freezer)
  cgroup on /sys/fs/cgroup/blkio type cgroup (rw,nosuid,nodev,noexec,relatime,blkio)
  cgroup on /sys/fs/cgroup/net_cls,net_prio type cgroup (rw,nosuid,nodev,noexec,relatime,net_cls,net_prio)
  cgroup on /sys/fs/cgroup/memory type cgroup (rw,nosuid,nodev,noexec,relatime,memory)
  cgroup on /sys/fs/cgroup/hugetlb type cgroup (rw,nosuid,nodev,noexec,relatime,hugetlb)
  cgroup on /sys/fs/cgroup/cpu,cpuacct type cgroup (rw,nosuid,nodev,noexec,relatime,cpu,cpuacct)
  cgroup on /sys/fs/cgroup/pids type cgroup (rw,nosuid,nodev,noexec,relatime,pids)
  cgroup on /sys/fs/cgroup/devices type cgroup (rw,nosuid,nodev,noexec,relatime,devices)
  cgroup on /sys/fs/cgroup/perf_event type cgroup (rw,nosuid,nodev,noexec,relatime,perf_event)
  cgroup on /sys/fs/cgroup/cpuset type cgroup (rw,nosuid,nodev,noexec,relatime,cpuset)
  mqueue on /dev/mqueue type mqueue (rw,relatime)
  hugetlbfs on /dev/hugepages type hugetlbfs (rw,relatime)
  systemd-1 on /proc/sys/fs/binfmt_misc type autofs (rw,relatime,fd=31,pgrp=1,timeout=0,minproto=5,maxproto=5,direct)
  debugfs on /sys/kernel/debug type debugfs (rw,relatime)
  fusectl on /sys/fs/fuse/connections type fusectl (rw,relatime)
  rpool/home on /home type zfs (rw,nosuid,noatime,xattr,noacl)
  rpool/home/fredrik on /home/fredrik type zfs (rw,nosuid,noatime,xattr,noacl)
  rpool/home/root on /root type zfs (rw,nosuid,noatime,xattr,noacl)
  rpool/srv on /srv type zfs (rw,noatime,xattr,noacl)
  rpool/var/cache on /var/cache type zfs (rw,nosuid,noexec,noatime,xattr,noacl)
  rpool/var/log on /var/log type zfs (rw,nosuid,noexec,noatime,xattr,noacl)
  rpool/var/spool on /var/spool type zfs (rw,nosuid,noexec,noatime,xattr,noacl)
  rpool/var/tmp on /var/tmp type zfs (rw,nosuid,noatime,xattr,noacl)
  tmpfs on /run/user/1000 type tmpfs (rw,nosuid,nodev,relatime,size=1623576k,mode=700,uid=1000,gid=1000)
  binfmt_misc on /proc/sys/fs/binfmt_misc type binfmt_misc (rw,relatime)
  tracefs on /sys/kernel/debug/tracing type tracefs (rw,relatime)

  cat /proc/mounts 
  sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0
  proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0
  udev /dev devtmpfs rw,nosuid,relatime,size=8066020k,nr_inodes=2016505,mode=755 0 0
  devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0
  tmpfs /run tmpfs rw,nosuid,noexec,relatime,size=1623576k,mode=755 0 0
  rpool/ROOT/ubuntu / zfs rw,relatime,xattr,noacl 0 0
  securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0
  tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0
  tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0
  tmpfs /sys/fs/cgroup tmpfs ro,nosuid,nodev,noexec,mode=755 0 0
  cgroup /sys/fs/cgroup/systemd cgroup rw,nosuid,nodev,noexec,relatime,xattr,release_agent=/lib/systemd/systemd-cgroups-agent,name=systemd 0 0
  pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0
  cgroup /sys/fs/cgroup/freezer cgroup rw,nosuid,nodev,noexec,relatime,freezer 0 0
  cgroup /sys/fs/cgroup/blkio cgroup rw,nosuid,nodev,noexec,relatime,blkio 0 0
  cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0
  cgroup /sys/fs/cgroup/memory cgroup rw,nosuid,nodev,noexec,relatime,memory 0 0
  cgroup /sys/fs/cgroup/hugetlb cgroup rw,nosuid,nodev,noexec,relatime,hugetlb 0 0
  cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0
  cgroup /sys/fs/cgroup/pids cgroup rw,nosuid,nodev,noexec,relatime,pids 0 0
  cgroup /sys/fs/cgroup/devices cgroup rw,nosuid,nodev,noexec,relatime,devices 0 0
  cgroup /sys/fs/cgroup/perf_event cgroup rw,nosuid,nodev,noexec,relatime,perf_event 0 0
  cgroup /sys/fs/cgroup/cpuset cgroup rw,nosuid,nodev,noexec,relatime,cpuset 0 0
  mqueue /dev/mqueue mqueue rw,relatime 0 0
  hugetlbfs /dev/hugepages hugetlbfs rw,relatime 0 0
  systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=31,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0
  debugfs /sys/kernel/debug debugfs rw,relatime 0 0
  fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0
  rpool/home /home zfs rw,nosuid,noatime,xattr,noacl 0 0
  rpool/home/fredrik /home/fredrik zfs rw,nosuid,noatime,xattr,noacl 0 0
  rpool/home/root /root zfs rw,nosuid,noatime,xattr,noacl 0 0
  rpool/srv /srv zfs rw,noatime,xattr,noacl 0 0
  rpool/var/cache /var/cache zfs rw,nosuid,noexec,noatime,xattr,noacl 0 0
  rpool/var/log /var/log zfs rw,nosuid,noexec,noatime,xattr,noacl 0 0
  rpool/var/spool /var/spool zfs rw,nosuid,noexec,noatime,xattr,noacl 0 0
  rpool/var/tmp /var/tmp zfs rw,nosuid,noatime,xattr,noacl 0 0
  tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=1623576k,mode=700,uid=1000,gid=1000 0 0
  binfmt_misc /proc/sys/fs/binfmt_misc binfmt_misc rw,relatime 0 0
  tracefs /sys/kernel/debug/tracing tracefs rw,relatime 0 0

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/initramfs-tools/+bug/1661629/+subscriptions