aufs based upgrade tests

[Michael Vogt]

Hi,

during the last UDS we talked informally about using the "aufs"
overlay filesystem layer for release upgrade testing. I build a
prototype implementation of this now that should be ok for public
testing. 

The idea discussed with Evan Dandrea (and others) was to create a
writable overlay into /tmp on top the systemdirs in "/" and then run
the release upgrade. This way we can test easily if the system would
upgrade cleanly (if no dpkg errors/maintainer script failures
happen). All writes go into /tmp so after the upgrade and on the next
reboot the system is back to its pre-upgraded state again (modulo
/home, that is not overlayed). It also means the next boot takes a
*long* time to clean /tmp - when I did test it on one of my production
machines that wait made me *really* nervous :) But its ok, it just takes
long (up to ~20 minutes or so).

Feedback is welcome, here is how to enable it:
$ sudo -s
# mkdir -p /etc/update-manager/release-upgrades.d/
# echo "[Aufs]" > /etc/update-manager/release-upgrades.d/aufs.cfg
# echo "EnableFullOverlay=yes" >> /etc/update-manager/release-upgrades.d/aufs.cfg
# exit
$ update-manager -d 
(or do-release-upgrade -d)

then follow the upgrade procedure noramlly. You will see a message
dialog saying "Sandbox mode". If you don't see that, something is
wrong and you need to cancel the upgrade (no worries, just hit
"cancel" in the final confirmation screen). When the final
confirmation screen appears, check with "mount" on a terminal if there
are really "aufs" mounts. Remember to remove the file (aufs.cfg) after
the next reboot if you want to perform a regular upgrade (otherwise
the next run will be a sandbox upgrade again :)

Caveats:
- it got *very little testing*, so be careful and have backups
- it does not work with the intrepid server kernel, it lacks the aufs
  module 
- the free space calculation is wrong (too small) when aufs is
  enabled, better have plenty of free space in /tmp (~4gb for 
  ubuntu-desktop)
- the system feels slow during the upgrade (more so than normal 
  due to aufs it seems, may also need more RAM, but I'm not sure about
  this yet) 
- the next reboot will take *ages* (because /tmp will be cleared from
  all the files written during the upgrade)
- there is a problem with some packages (libthunar-vfs-1-2 is one)
  where dpkg return "Invalid cross-device link" error. it seems
  like its a combination of aufs/dpkg problem and currently under
  investigation


Cheers,
 Michael