[Bug 1089195] Re: linux-headers will eat your inodes on LTS.
algal
1089195 at bugs.launchpad.net
Sat Feb 6 06:08:09 UTC 2016
I have just encountered this issue on an Ubuntu 14.04 LTS instance. It
was setup in early 2014, and has had only a light workload serving an
API backend for a medical education system in use by many doctors.
Ironically, it seems to be _because_ I enabled unattended-upgrades that
the system now cannot update itself.
And I not sure how to repair this situation without interrupting
service, since many package management commands depend on first doing
apt-get update, which is what I cannot do. And I don't think I
understand the system well enough to be sure that if I erase something
manually it is not critical.
If anyone has advice on this, I'd appreciate, even if only pointers to
books on more esoteric aspects of Ubuntu package management.
$ lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description: Ubuntu 12.04.4 LTS
Release: 12.04
Codename: precise
$ df -h
Filesystem Size Used Avail Use% Mounted on
/dev/xvda1 7.9G 5.9G 1.7G 78% /
udev 288M 12K 287M 1% /dev
tmpfs 60M 196K 59M 1% /run
none 5.0M 0 5.0M 0% /run/lock
none 296M 0 296M 0% /run/shm
$ df -i
Filesystem Inodes IUsed IFree IUse% Mounted on
/dev/xvda1 524288 518210 6078 99% /
udev 73475 379 73096 1% /dev
tmpfs 75541 266 75275 1% /run
none 75541 3 75538 1% /run/lock
none 75541 1 75540 1% /run/shm
$ find /usr/src/ -type f | wc -l
345657
$ uptime
06:18:41 up 664 days, 11:16, 2 users, load average: 0.21, 0.20, 0.13
--
You received this bug notification because you are a member of Ubuntu
Foundations Bugs, which is subscribed to update-manager in Ubuntu.
https://bugs.launchpad.net/bugs/1089195
Title:
linux-headers will eat your inodes on LTS.
Status in update-manager package in Ubuntu:
Confirmed
Bug description:
Hello all,
Summary
-----------------
Both linux-image-* and linux-headers-* are installed every time you upgrade the kernel. However, they are never removed by any maintenance process.
Every linux-headers-*-generic-pae package has approx. 6,700 files on
it. Regular headers packages have even more files: around 11,700 files
each.
Although these packages' files won't occupy much space, after some
years (think LTS installation), they will "eat" all the inodes on your
root partition.
The first effect you'll encounter will be the you are unable to
upgrade your system.
This is a situation that will affect all users, however it will be a
greater problem regular non-technical user.
There's no simple, high-level tool to solve this problem.
Case Study
---------------
I have a 2-year-and-8-months old 10.04 installation. I have a ~10GB root partition, which is double of the minimum recommend (5GB).
$ df -h --type=ext4
Filesystem Size Used Avail Use% Mounted on
/dev/sda5 9.4G 6.2G 2.8G 70% /
/dev/sda6 94G 45G 45G 51% /home
It looks that I had plenty of space left to upgrade, but I got this
error while upgrading:
unable to create `/usr/src/linux-headers-2.6.32-45/arch/s390/include/asm/nmi.h.dpkg-new'
(while processing `./usr/src/linux-headers-2.6.32-45/arch/s390/include/asm/nmi.h'): No space left on device
It was because of I was running out of inodes:
$ df -i /
Filesystem Inodes IUsed IFree IUse% Mounted on
/dev/sda5 625856 618015 7841 99% /
Why? Because the huge amount of linux-headers files:
$ find /usr/src -type f | wc -l
355112
That is more than three hundred and fifty thousand files!
These are the packages that I removed:
Purg linux-headers-2.6.32-21 [2.6.32-21.32]
Purg linux-headers-2.6.32-22-generic-pae [2.6.32-22.36]
Purg linux-headers-2.6.32-22 [2.6.32-22.36]
Purg linux-headers-2.6.32-23-generic-pae [2.6.32-23.37]
Purg linux-headers-2.6.32-23 [2.6.32-23.37]
Purg linux-headers-2.6.32-24-generic-pae [2.6.32-24.43]
Purg linux-headers-2.6.32-24 [2.6.32-24.43]
Purg linux-headers-2.6.32-25-generic-pae [2.6.32-25.45]
Purg linux-headers-2.6.32-25 [2.6.32-25.45]
Purg linux-headers-2.6.32-26-generic-pae [2.6.32-26.48]
Purg linux-headers-2.6.32-26 [2.6.32-26.48]
Purg linux-headers-2.6.32-27-generic-pae [2.6.32-27.49]
Purg linux-headers-2.6.32-27 [2.6.32-27.49]
Purg linux-headers-2.6.32-28-generic-pae [2.6.32-28.55]
Purg linux-headers-2.6.32-28 [2.6.32-28.55]
Purg linux-headers-2.6.32-29-generic-pae [2.6.32-29.58]
Purg linux-headers-2.6.32-29 [2.6.32-29.58]
Purg linux-headers-2.6.32-30-generic-pae [2.6.32-30.59]
Purg linux-headers-2.6.32-30 [2.6.32-30.59]
Purg linux-headers-2.6.32-31-generic-pae [2.6.32-31.61]
Purg linux-headers-2.6.32-31 [2.6.32-31.61]
Purg linux-headers-2.6.32-32-generic-pae [2.6.32-32.62]
Purg linux-headers-2.6.32-32 [2.6.32-32.62]
Purg linux-headers-2.6.32-33-generic-pae [2.6.32-33.72]
Purg linux-headers-2.6.32-33 [2.6.32-33.72]
Purg linux-headers-2.6.32-34-generic-pae [2.6.32-34.77]
Purg linux-headers-2.6.32-34 [2.6.32-34.77]
Purg linux-headers-2.6.32-35-generic-pae [2.6.32-35.78]
Purg linux-headers-2.6.32-35 [2.6.32-35.78]
Purg linux-headers-2.6.32-36-generic-pae [2.6.32-36.79]
Purg linux-headers-2.6.32-36 [2.6.32-36.79]
Purg linux-headers-2.6.32-37-generic-pae [2.6.32-37.81]
Purg linux-headers-2.6.32-37 [2.6.32-37.81]
Purg linux-headers-2.6.32-38-generic-pae [2.6.32-38.83]
Purg linux-headers-2.6.32-38 [2.6.32-38.83]
Purg linux-headers-2.6.32-39-generic-pae [2.6.32-39.86]
Purg linux-headers-2.6.32-39 [2.6.32-39.86]
Purg linux-headers-2.6.32-40-generic-pae [2.6.32-40.87]
Purg linux-headers-2.6.32-40 [2.6.32-40.87]
Purg linux-headers-2.6.32-41-generic-pae [2.6.32-41.94]
Purg linux-headers-2.6.32-41 [2.6.32-41.94]
Purg linux-headers-2.6.32-42-generic-pae [2.6.32-42.96]
Purg linux-headers-2.6.32-42 [2.6.32-42.96]
Purg linux-headers-2.6.32-43-generic-pae [2.6.32-43.97]
Purg linux-headers-2.6.32-43 [2.6.32-43.97]
Purg linux-headers-generic-pae [2.6.32.45.52]
Purg linux-image-2.6.32-42-generic-pae [2.6.32-42.96]
Purg linux-image-2.6.32-43-generic-pae [2.6.32-43.97]
Then, problem solved:
$ find /usr/src/ -type f | wc -l
28276
$ ls /usr/src/
linux-headers-2.6.32-44 linux-headers-2.6.32-45-generic-pae
linux-headers-2.6.32-44-generic-pae nvidia-current-195.36.24
linux-headers-2.6.32-45 virtualbox-ose-3.1.6
$ df -i --type=ext4
Filesystem Inodes IUsed IFree IUse% Mounted on
/dev/sda5 625856 192891 432965 31% /
/dev/sda6 6225920 95025 6130895 2% /home
To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/update-manager/+bug/1089195/+subscriptions
More information about the foundations-bugs
mailing list