[Bug 1667512] Re: update-initramfs hangs on upgrade, dpkg unusable, unbootable system
mesiu84
1667512 at bugs.launchpad.net
Thu Mar 9 09:54:57 UTC 2017
Have the same problem since yesterday. I'm having 4.4.0-47 4.4.0-59
4.4.0-62 4.4.0-64 installed. I was trying to upgrade my system yesterday
and 4.4.0-66 was about to be installed, but it hangs on update-initramfs
or to be more precise it's hanging on "Building cpio" when running
update-initramfs -u -v
Problem is related probably to last 2 upgrades which I did.
Here is the list of packages installed yesterday (aptitude):
[REMOVE, NOT USED] ubuntu-core-launcher:amd64 2.22.6
[UPGRADE] snapd:amd64 2.22.3 -> 2.22.6
[UNCONFIGURED] linux-generic:amd64 4.4.0.66.70
[UNCONFIGURED] linux-headers-4.4.0-66:amd64 4.4.0-66.87
[UNCONFIGURED] linux-headers-4.4.0-66-generic:amd64 4.4.0-66.87
[UNCONFIGURED] linux-headers-generic:amd64 4.4.0.66.70
[UNCONFIGURED] linux-image-4.4.0-66-generic:amd64 4.4.0-66.87
[UNCONFIGURED] linux-image-extra-4.4.0-66-generic:amd64 4.4.0-66.87
[UNCONFIGURED] linux-image-generic:amd64 4.4.0.66.70
[UNCONFIGURED] linux-libc-dev:amd64 4.4.0-66.87
[UNCONFIGURED] lxd:amd64 2.0.9-0ubuntu1~16.04.2
[UNCONFIGURED] lxd-client:amd64 2.0.9-0ubuntu1~16.04.2
[UNCONFIGURED] mdadm:amd64 3.3-2ubuntu7.2
[UNCONFIGURED] snap-confine:amd64 2.22.6
[UNCONFIGURED] ubuntu-snappy-cli:amd64 2.22.6
here is the list of previous upgrade (apt):
Start-Date: 2017-02-21 23:48:52
Commandline: apt upgrade
Install: linux-headers-4.4.0-64:amd64 (4.4.0-64.85, automatic), linux-image-4.4.0-64-generic:amd64 (4.4.0-64.85, automatic), linux-image-extra-4.4.0-64-generic:amd64 (4.4.0-64.85, automatic), linux-headers-4.4.0-64-generic:amd64 (4.4.0-64.85, automatic)
Upgrade: lxc-common:amd64 (2.0.6-0ubuntu1~ubuntu16.04.2, 2.0.7-0ubuntu1~16.04.1), libssh2-1:amd64 (1.5.0-2, 1.5.0-2ubuntu0.1), libdns-export162:amd64 (1:9.10.3.dfsg.P4-8ubuntu1.4, 1:9.10.3.dfsg.P4-8ubuntu1.5), libisccfg140:amd64 (1:9.10.3.dfsg.P4-8ubuntu1.4, 1:9.10.3.dfsg.P4-8ubuntu1.5), linux-headers-generic:amd64 (4.4.0.62.65, 4.4.0.64.68), linux-libc-dev:amd64 (4.4.0-62.83, 4.4.0-64.85), linux-image-generic:amd64 (4.4.0.62.65, 4.4.0.64.68), snapd:amd64 (2.21, 2.22.3), bind9-host:amd64 (1:9.10.3.dfsg.P4-8ubuntu1.4, 1:9.10.3.dfsg.P4-8ubuntu1.5), snap-confine:amd64 (2.21, 2.22.3), libisc160:amd64 (1:9.10.3.dfsg.P4-8ubuntu1.4, 1:9.10.3.dfsg.P4-8ubuntu1.5), lxd:amd64 (2.0.8-0ubuntu1~ubuntu16.04.2, 2.0.9-0ubuntu1~16.04.1), liblxc1:amd64 (2.0.6-0ubuntu1~ubuntu16.04.2, 2.0.7-0ubuntu1~16.04.1), libisc-export160:amd64 (1:9.10.3.dfsg.P4-8ubuntu1.4, 1:9.10.3.dfsg.P4-8ubuntu1.5), liblwres141:amd64 (1:9.10.3.dfsg.P4-8ubuntu1.4, 1:9.10.3.dfsg.P4-8ubuntu1.5), ubuntu-core-launcher:amd64 (2.21, 2.22.3), libdns162:amd64 (1:9.10.3.dfsg.P4-8ubuntu1.4, 1:9.10.3.dfsg.P4-8ubuntu1.5), ubuntu-snappy-cli:amd64 (2.21, 2.22.3), libisccc140:amd64 (1:9.10.3.dfsg.P4-8ubuntu1.4, 1:9.10.3.dfsg.P4-8ubuntu1.5), libbind9-140:amd64 (1:9.10.3.dfsg.P4-8ubuntu1.4, 1:9.10.3.dfsg.P4-8ubuntu1.5), linux-generic:amd64 (4.4.0.62.65, 4.4.0.64.68), lxd-client:amd64 (2.0.8-0ubuntu1~ubuntu16.04.2, 2.0.9-0ubuntu1~16.04.1), lxcfs:amd64 (2.0.5-0ubuntu1~ubuntu16.04.1, 2.0.6-0ubuntu1~16.04.1)
End-Date: 2017-02-21 23:49:31
I'm able to update initrd.img file using following command, it doesnt hang on that "Building cpio":
mkinitramfs -v -c -k 4.4.0-66-generic -o /tmp/initrd.img.tmp
I'm not planning to restart server during next few weeks, so I'm unable
to check if it's running. Right now I'm working on 4.4.0-62.
--
You received this bug notification because you are a member of Ubuntu
Foundations Bugs, which is subscribed to initramfs-tools in Ubuntu.
https://bugs.launchpad.net/bugs/1667512
Title:
update-initramfs hangs on upgrade, dpkg unusable, unbootable system
Status in initramfs-tools package in Ubuntu:
Confirmed
Bug description:
At least four users, including myself, are having an issue with
update-initramfs hanging while updating ubuntu 16.04. The bug has been
documented while attempting an update to multiple kernel versions (
4.4.0-24, 4.4.0-62, 4.4.0-63). The bug causes any apt-get update or
install to fail, and may also lead to an unbootable system.
User #1 (me):
$ uname -a
Linux <computername> 4.4.0-62-generic #83-Ubuntu SMP Wed Jan 18 14:10:15 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux
$ sudo apt-get upgrade
Fetched 1,571 MB in 2min 9s (12.2 MB/s)
Extracting templates from packages: 100%
Preconfiguring packages ...
(Reading database ... 344634 files and directories currently installed.)
Preparing to unpack .../base-files_9.4ubuntu4.4_amd64.deb ...
Unpacking base-files (9.4ubuntu4.4) over (9.4ubuntu4.3) ...
Processing triggers for plymouth-theme-ubuntu-text (0.9.2-3ubuntu13.1) ...
update-initramfs: deferring update (trigger activated)
Processing triggers for man-db (2.7.5-1) ...
Processing triggers for cracklib-runtime (2.9.2-1build2) ...
Processing triggers for install-info (6.1.0.dfsg.1-5) ...
Processing triggers for initramfs-tools (0.122ubuntu8.8) ...
update-initramfs: Generating /boot/initrd.img-4.4.0-63-generic
<hangs overnight> Uh oh! Let's try to troubleshoot.
$ sudo killall dpkg
$ sudo dpkg --configure -a
Setting up initramfs-tools (0.122ubuntu8.8) ...
update-initramfs: deferring update (trigger activated)
Setting up base-files (9.4ubuntu4.4) ...
Installing new version of config file /etc/issue ...
Installing new version of config file /etc/issue.net ...
Installing new version of config file /etc/lsb-release ...
Setting up linux-image-4.4.0-63-generic (4.4.0-63.84) ...
Running depmod.
update-initramfs: deferring update (hook will be called later)
initrd.img(/boot/initrd.img-4.4.0-63-generic
) points to /boot/initrd.img-4.4.0-63-generic
(/boot/initrd.img-4.4.0-63-generic) -- doing nothing at /var/lib/dpkg/info/linux-image-4.4.0-63-generic.postinst line 491.
vmlinuz(/boot/vmlinuz-4.4.0-63-generic
) points to /boot/vmlinuz-4.4.0-63-generic
(/boot/vmlinuz-4.4.0-63-generic) -- doing nothing at /var/lib/dpkg/info/linux-image-4.4.0-63-generic.postinst line 491.
Examining /etc/kernel/postinst.d.
run-parts: executing /etc/kernel/postinst.d/apt-auto-removal 4.4.0-63-generic /boot/vmlinuz-4.4.0-63-generic
run-parts: executing /etc/kernel/postinst.d/dkms 4.4.0-63-generic /boot/vmlinuz-4.4.0-63-generic
run-parts: executing /etc/kernel/postinst.d/initramfs-tools 4.4.0-63-generic /boot/vmlinuz-4.4.0-63-generic
update-initramfs: Generating /boot/initrd.img-4.4.0-63-generic
^CFailed to process /etc/kernel/postinst.d at /var/lib/dpkg/info/linux-image-4.4.0-63-generic.postinst line 1052.
dpkg: error processing package linux-image-4.4.0-63-generic (--configure):
subprocess installed post-installation script returned error exit status 2
dpkg: dependency problems prevent configuration of linux-image-generic:
linux-image-generic depends on linux-image-4.4.0-63-generic; however:
Package linux-image-4.4.0-63-generic is not configured yet.
dpkg: error processing package linux-image-generic (--configure):
dependency problems - leaving unconfigured
dpkg: dependency problems prevent configuration of linux-signed-image-4.4.0-63-generic:
linux-signed-image-4.4.0-63-generic depends on linux-image-4.4.0-63-generic (= 4.4.0-63.84); however:
Package linux-image-4.4.0-63-generic is not configured yet.
dpkg: error processing package linux-signed-image-4.4.0-63-generic (--configure):
dependency problems - leaving unconfigured
dpkg: dependency problems prevent configuration of linux-signed-image-generic:
linux-signed-image-generic depends on linux-signed-image-4.4.0-63-generic; however:
Package linux-signed-image-4.4.0-63-generic is not configured yet.
dpkg: error processing package linux-signed-image-generic (--configure):
dependency problems - leaving unconfigured
dpkg: dependency problems prevent configuration of linux-image-extra-4.4.0-63-generic:
linux-image-extra-4.4.0-63-generic depends on linux-image-4.4.0-63-generic; however:
Package linux-image-4.4.0-63-generic is not configured yet.
dpkg: error processing package linux-image-extra-4.4.0-63-generic (--configure):
dependency problems - leaving unconfigured
dpkg: dependency problems prevent configuration of linux-generic:
linux-generic depends on linux-image-generic (= 4.4.0.63.67); however:
Package linux-image-generic is not configured yet.
dpkg: error processing package linux-generic (--configure):
dependency problems - leaving unconfigured
dpkg: dependency problems prevent configuration of linux-signed-generic:
linux-signed-generic depends on linux-signed-image-generic (= 4.4.0.63.67); however:
Package linux-signed-image-generic is not configured yet.
dpkg: error processing package linux-signed-generic (--configure):
dependency problems - leaving unconfigured
Processing triggers for initramfs-tools (0.122ubuntu8.8) ...
update-initramfs: Generating /boot/initrd.img-4.4.0-63-generic
^Cdpkg: error processing package initramfs-tools (--configure):
subprocess installed post-installation script was interrupted
Processing triggers for plymouth-theme-ubuntu-text (0.9.2-3ubuntu13.1) ...
update-initramfs: deferring update (trigger activated)
Errors were encountered while processing:
linux-image-4.4.0-63-generic
linux-image-generic
linux-signed-image-4.4.0-63-generic
linux-signed-image-generic
linux-image-extra-4.4.0-63-generic
linux-generic
linux-signed-generic
initramfs-tools
I verified that /boot has enough space:
/dev/nvme0n1p2 496M 314M 157M 67% /boot
Here's my /boot:
$ ls -l /boot
total 325378
-rw-r--r-- 1 root root 1244118 Jan 6 16:44 abi-4.4.0-59-generic
-rw-r--r-- 1 root root 1244118 Jan 18 07:59 abi-4.4.0-62-generic
-rw-r--r-- 1 root root 1245512 Feb 1 11:39 abi-4.4.0-63-generic
-rw-r--r-- 1 root root 1245512 Feb 20 05:40 abi-4.4.0-64-generic
-rw-r--r-- 1 root root 190047 Jan 6 16:44 config-4.4.0-59-generic
-rw-r--r-- 1 root root 190047 Jan 18 07:59 config-4.4.0-62-generic
-rw-r--r-- 1 root root 190247 Feb 1 11:39 config-4.4.0-63-generic
-rw-r--r-- 1 root root 190247 Feb 20 05:40 config-4.4.0-64-generic
drwx------ 3 root root 4096 Dec 31 1969 efi
drwxr-xr-x 5 root root 1024 Feb 23 13:23 grub
-rw-r--r-- 1 root root 10931153 Feb 23 13:56 initrd.img-4.4.0-57-generic
-rw-r--r-- 1 root root 46506399 Feb 9 05:37 initrd.img-4.4.0-59-generic
-rw-r--r-- 2 root root 39526162 Feb 23 13:27 initrd.img-4.4.0-62-generic
-rw-r--r-- 2 root root 39526162 Feb 23 13:27 initrd.img-4.4.0-62-generic.dpkg-bak
-rw-r--r-- 1 root root 0 Feb 23 13:56 initrd.img-4.4.0-62-generic.new
-rw-r--r-- 2 root root 26019417 Feb 23 13:11 initrd.img-4.4.0-63-generic
-rw-r--r-- 2 root root 26019417 Feb 23 13:11 initrd.img-4.4.0-63-generic.dpkg-bak
-rw-r--r-- 1 root root 0 Feb 23 13:56 initrd.img-4.4.0-63-generic.new
-rw-r--r-- 1 root root 39516686 Feb 23 14:50 initrd.img-4.4.0-64-generic
-rw-r--r-- 1 root root 39516505 Feb 23 13:33 initrd.img-4.4.0-64-generic.dpkg-bak
drwx------ 2 root root 12288 Jul 31 2016 lost+found
-rw-r--r-- 1 root root 182704 Jan 28 2016 memtest86+.bin
-rw-r--r-- 1 root root 184380 Jan 28 2016 memtest86+.elf
-rw-r--r-- 1 root root 184840 Jan 28 2016 memtest86+_multiboot.bin
-rw------- 1 root root 3875594 Jan 6 16:44 System.map-4.4.0-59-generic
-rw------- 1 root root 3875553 Jan 18 07:59 System.map-4.4.0-62-generic
-rw------- 1 root root 3883990 Feb 1 11:39 System.map-4.4.0-63-generic
-rw------- 1 root root 3883990 Feb 20 05:40 System.map-4.4.0-64-generic
-rw------- 1 root root 7069136 Jan 6 16:44 vmlinuz-4.4.0-59-generic
-rw------- 1 root root 7071064 Jan 11 07:08 vmlinuz-4.4.0-59-generic.efi.signed
-rw------- 1 root root 7070992 Jan 18 07:59 vmlinuz-4.4.0-62-generic
-rw------- 1 root root 7072920 Feb 3 08:04 vmlinuz-4.4.0-62-generic.efi.signed
-rw------- 1 root root 7087088 Feb 1 11:39 vmlinuz-4.4.0-63-generic
-rw------- 1 root root 7087152 Feb 20 05:40 vmlinuz-4.4.0-64-generic
After uninstall of linux-image-4.4.0-63-generic:
$ sudo dpkg --status linux-image-4.4.0-63-genericPackage: linux-image-4.4.0-63-generic
Status: deinstall ok config-files
Priority: optional
Section: kernel
Installed-Size: 64800
Maintainer: Ubuntu Kernel Team <kernel-team at lists.ubuntu.com>
Architecture: amd64
Source: linux
Version: 4.4.0-63.84
Provides: fuse-module, ivtv-modules, kvm-api-4, linux-image, redhat-cluster-modules, spl-dkms, virtualbox-guest-modules, zfs-dkms
Depends: initramfs-tools | linux-initramfs-tool, kmod
Recommends: grub-pc | grub-efi-amd64 | grub-efi-ia32 | grub | lilo
Suggests: fdutils, linux-doc-4.4.0 | linux-source-4.4.0, linux-tools, linux-headers-4.4.0-63-generic
There does seem to be a problem with linux-image-
extra-4.4.0-63-generic. Trying to uninstall this triggers update-
initramfs.
$ sudo dpkg --status linux-image-extra-4.4.0-63-generic
Package: linux-image-extra-4.4.0-63-generic
Status: purge ok half-installed
Priority: optional
Section: kernel
Installed-Size: 148939
Maintainer: Ubuntu Kernel Team <kernel-team at lists.ubuntu.com>
Architecture: amd64
Source: linux
Version: 4.4.0-63.84
Depends: linux-image-4.4.0-63-generic, crda | wireless-crda
I've tried http://askubuntu.com/questions/650732/apt-fails-to-remove-
partially-installed-kernel-and-cant-install-any-other-packa/651596
with no luck. At this point I'm not sure if a partially installed
kernel is the problem, and I've run out of troubleshooting insight.
User #2:
https://ubuntuforums.org/showthread.php?t=2351895
User #3:
https://ubuntuforums.org/showthread.php?t=2353606
User #4 (original question):
When upgrading (apt-get upgrade) on 16.04 (ubuntu-mate) I get a hang
(left from 9:50am til 16:00 or thereabouts)
I tracked down as far as the call to /etc/kernel/postinst.d/initramfs-
tools and added a -v option, the last relevant lines on the terminal
when re-running `apt-get upgrade` are as follows:
...
Adding binary /bin/date
Adding binary /sbin/hwclock
Adding binary /sbin/dumpe2fs
Calling hook dmsetup
Calling hook reiserfsprogs
Building cpio /boot/initrd.img-4.4.0-24-generic.new initramfs
^CFailed to process /etc/kernel/postinst.d at /var/lib/dpkg/info/linux-image-4.4.0-24-generic.postinst line 1052.
dpkg: error processing package linux-image-4.4.0-24-generic (--configure):
subprocess installed post-installation script returned error exit status 2
dpkg: dependency problems prevent configuration of linux-image-extra-4.4.0-24-generic:
linux-image-extra-4.4.0-24-generic depends on linux-image-4.4.0-24-generic; however:
Package linux-image-4.4.0-24-generic is not configured yet.
Checking with `ps fax` before the interrupt, I get this tree - indicating a hang on `sync` task
sudo dpkg --configure -a
\_ dpkg --configure -a
\_ /usr/bin/perl /var/lib/dpkg/info/linux-image-4.4.0-24-generic.postinst configure
\_ run-parts --verbose --exit-on-error --arg=4.4.0-24-generic --arg=/boot/vmlinuz-4.4.0-24-generic /etc/kernel/postinst.d
\_ /bin/sh -e /etc/kernel/postinst.d/initramfs-tools 4.4.0-24-generic /boot/vmlinuz-4.4.0-24-generic
\_ /bin/sh /usr/sbin/update-initramfs -v -c -t -k 4.4.0-24-generic -b /boot
\_ sync
I tried moving the resulting initramfs img file in /boot so a new one
would be created - a new one is indeed created, hang still occurs on
`sync`
To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/initramfs-tools/+bug/1667512/+subscriptions
More information about the foundations-bugs
mailing list