Testing of filesystem identification code (blkid/vol_id)

[Scott James Remnant]

I'd appreciate some testing of the following packages in my PPA:

	deb http://ppa.launchpad.net/scott/ppa/ubuntu jaunty main
	deb-src http://ppa.launchpad.net/scott/ppa/ubuntu jaunty main

	<https://launchpad.net/~scott/+archive/ppa>

util-linux	2.15~rc1-1ubuntu1~blkid3
    libblkid1
    libblkid-dev
    bsdutils
    mount
    util-linux
    util-linux-locales

e2fsprogs	1.41.4-1ubuntu2~blkid1
    e2fsprogs
    e2fslibs
    libuuid1
    uuid-dev
    uuid-runtime
    libss2
    ss-dev
    libcomerr2
    comerr-dev
    e2fsck-static

udev		140-2~blkid1
    udev
    libudev0
    libudev-dev

devmapper	2:1.02.27-4ubuntu5~blkid1
    dmsetup
    dmeventd
    libdevmapper1.02.1
    libdevmapper-dev
    libdevmapper-event1.02.1

mdadm		2.6.7.1-1ubuntu8~blkid1
    mdadm

initramfs-tools	0.92bubuntu26~blkid1
    initramfs-tools	


Obviously, only upgrade those you have installed.  All the version
numbers are greater than in jaunty at the time of mailing, so APT should
do the right thing.

If your computer works after a reboot, then everything's OK :-)


Background:

Up until now, we've had two different libraries and tools for examining
the content of block devices and finding out information about the
filesystem contained therein.  Most particularly, the UUID, LABEL and
TYPE of that filesystem.

The first and oldest of these is blkid/libblkid and comes with the
e2fsprogs source package, which contains the tools for the ext2, etc.
filesystem.

The second and newest of these is vol_id/libvolume-id and comes with the
udev source package.

Why we have two at all is a bit of a story, but fundamentally both
libraries behave slightly differently and the maintainers never
persuaded each other to adopt or integrate the other view.

blkid is designed to answer the question "which filesystem has UUID or
LABEL foo?".  It keeps an on-disk cache of known mappings, and if you
ask for a mapping it doesn't know, it scans every block device on the
system looking for it.

vol_id is designed to answer the question "what is in this block
device?".  You give it a known block device, and it tells you what's in
it.


At first, this might seem ok, except that both have different levels of
filesystem coverage (vol_id has more wide coverage, including of RAID
and LVM; blkid knows about different versions of ext4).  Not only that,
but sometimes a block device will have the metadata of two different
filesystems (due to a bad format), and each might give a different
answer as to which is the right one.

So we should just pick one, right?

That's what we've been doing up until now, but it hasn't gone quite
smoothly.


Since udev knows when devices are added to and removed from the system,
manages the /dev tree, and runs programs such as LVM and mdadm, it also
logically probes the filesystem content.

Thus it needs a tool that behaves like vol_id (indeed, this is why
vol_id was written and why it's in the udev package).  HAL also uses the
libvolume-id library for its probing.

But vol_id doesn't provide the reverse lookup, however udev uses its
information to maintain the /dev/disk/by-uuid and /dev/disk/by-label
directories of symlinks.

So blkid UUID and LABEL lookups could be replaced by filesystem path
lookups.

That's what we've done until now; our versions of the tools in
util-linux (mount, swapon, etc.) were patched to translate UUID= and
LABEL= lookups into /dev/disk/by-{uuid,label}/* path lookups.

initramfs-tools did the same thing.

Perhaps irritatingly, our fsck tool comes from e2fsprogs and was not
patched.  Likewise initscripts called "findfs" which comes from blkid.

So we had some bugs where the two disagreed.

Also this has a shortcoming that the /dev/disk tree is only updated when
udev sees a change in the physical block device.  Running dd, mkfs,
mkswap, etc. on a block device does not update the symlinks, even though
the UUID and LABEL may have changed.


So there's been a number of parallel efforts to truly fix this.

Firstly udev now watches block devices; if they are written to, it acts
as if there has been a change to the physical block device.  Thus
the /dev/disk/by-uuid and /dev/disk/by-label symlinks *are* now changed
by dd, mkfs, mkswap, etc.

This is already in jaunty.


The next change is what's in the PPA.

The libblkid library from e2fsprogs and the libvolume-id library from
udev have been merged.  This new library, still called libblkid, and
backwards compatible in API with it, is part of util-linux-ng (our
util-linux source package).

This newly merged library supports the old libblkid cache API, and also
supports a probe a single block device API which is along similar lines
to the old libvolume-id API (but software[0] does need patching).

Along with the library, the blkid and findfs tools have been moved to
util-linux-ng as well.  blkid supports a new "-p" (probe) option for
probing a single block device.  Also when performing a simple UUID or
LABEL to device lookup, both tools now first check udev's symlinks
rather than scanning.

The various tools in util-linux-ng (mount, swapon, etc.) all use this
new library.

Finally fsck itself has been moved from e2fsprogs to util-linux, and
uses this new library as well.


So we now have a single filesystem probing library and tool.


Along with the updates to util-linux, my PPA contains updates to udev,
dmsetup and mdadm that change the rules to run "blkid" instead of
"vol_id".

It also contains an update to initramfs-tools to use blkid in all of the
places it previously used vol_id.  Mostly these are to lookup the UUID
of a given filesystem; but also in the root mounting code to check that
a block device contains a valid filesystem.


HAL has not yet been patched, it will still use libvolume-id.

Scott

[0] basically, only HAL
-- 
Scott James Remnant
scott at canonical.com
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 197 bytes
Desc: This is a digitally signed message part
Url : https://lists.ubuntu.com/archives/ubuntu-devel/attachments/20090324/ee575436/attachment.pgp