[Bug 688541] Re: race condition on shutdown (leads to corrupted fs)

[Clint Byrum]

** Changed in: mysql-5.5 (Ubuntu Precise)
       Status: In Progress => Invalid

** Changed in: mysql-5.1 (Ubuntu Precise)
       Status: Triaged => Invalid

** Changed in: sysvinit (Ubuntu Precise)
       Status: Triaged => In Progress

** Changed in: sysvinit (Ubuntu Precise)
     Assignee: Canonical Foundations Team (canonical-foundations) => Clint Byrum (clint-fewbar)

-- 
You received this bug notification because you are a member of Ubuntu
Foundations Bugs, which is subscribed to sysvinit in Ubuntu.
https://bugs.launchpad.net/bugs/688541

Title:
  race condition on shutdown (leads to corrupted fs)

Status in “mysql-5.1” package in Ubuntu:
  Invalid
Status in “mysql-5.5” package in Ubuntu:
  Invalid
Status in “sysvinit” package in Ubuntu:
  In Progress
Status in “mysql-5.1” source package in Precise:
  Invalid
Status in “mysql-5.5” source package in Precise:
  Invalid
Status in “sysvinit” source package in Precise:
  In Progress

Bug description:
  I'm using mysql-server-5.1 on a 10.04 LTS installation.
  The mysql db is around 27GB and on a separate partition mounted as /var/lib/mysql.

  On shutdown I get the following error message:

  Checking for running unattended-upgrades:  * Asking all remaining processes to terminate...       
  [80G 
  [74G[ OK ]
   * All processes ended within 1 seconds....       
  [80G 
  [74G[ OK ]
   * Deconfiguring network interfaces...       
  [80G 
  [74G[ OK ]
   * Deactivating swap...       
  [80G 
  [74G[ OK ]
   * Unmounting local filesystems...       
  [80G umount2: Device or resource busy
  umount: /var/lib/mysql: device is busy.
          (In some cases useful info about processes that use
           the device is found by lsof(8) or fuser(1))
  umount2: Device or resource busy
  umount2: Device or resource busy
  umount: /tmp: device is busy.
          (In some cases useful info about processes that use
           the device is found by lsof(8) or fuser(1))
  umount2: Device or resource busy
  [74G[
  [31mfail
  [39;49m]
  mount: / is busy
   * Will now restart
  [ 3369.429751] Restarting system.

  
  On the next reboot the file system is corrupt and need to be fsck-ed.

  I think the problem is, that mysql uses an upstart job (/etc/init/mysql.conf) and has
  stop on runlevel [016]

  The rc.conf job is also triggered on runlevel 0 and 6, so they
  basically run at the same time.As

  When /etc/rc0.d/S20sendsigs is run, it deliberatly does not wait or
  kill any upstart jobs.

  As my mysqld process takes some time to shutdown, S40umountfs and
  S60umountroot are run before the mysqld has quit.

  Leading to the fs not being properly unmounted. It is event possible
  that mysqld is forcefully killed by halt in S90halt if it hasn't
  stopped by then.

  This is a serious issue, as it can (and will) lead to data loss.

  Other upstart jobs, like rsyslog.conf, use the same "stop on runlevel
  [016]" stanza, so they are probably affected too.

  ProblemType: Bug
  DistroRelease: Ubuntu 10.10
  Package: mysql-server-5.1 5.1.49-1ubuntu8.1
  Uname: Linux 2.6.32-5-686 i686
  NonfreeKernelModules: michael_mic arc4 ecb lib80211_crypt_tkip aes_i586 aes_generic lib80211_crypt_ccmp sco bnep rfcomm l2cap binfmt_misc acpi_cpufreq ppdev lp cpufreq_userspace cpufreq_stats vboxnetadp cpufreq_powersave vboxnetflt cpufreq_conservative vboxdrv fuse pcmcia snd_intel8x0m snd_intel8x0 snd_ac97_codec btusb bluetooth rfkill ac97_bus yenta_socket ipw2200 snd_pcm 8139too firewire_ohci snd_seq 8139cp firewire_core sg uhci_hcd snd_timer rsrc_nonstatic libipw snd_seq_device pcmcia_core crc_itu_t parport_pc smsc_ircc2 ehci_hcd mii joydev lib80211 sr_mod parport i2c_i801 irda snd usbcore wbsd soundcore shpchp mmc_core pcspkr rng_core cdrom psmouse container crc_ccitt snd_page_alloc serio_raw pci_hotplug ac battery nls_base processor evdev ppp_generic slhc loop autofs4 ext4 mbcache jbd2 crc16 dm_mod sd_mod crc_t10dif radeon ttm ata_generic drm_kms_helper ata_piix drm i2c_algo_bit libata video thermal i2c_core scsi_mod output thermal_sys button
  Architecture: i386
  Date: Fri Dec 10 13:41:52 2010
  ProcEnviron:
   PATH=(custom, no user)
   LANG=de_DE.utf8
   SHELL=/bin/bash
  SourcePackage: mysql-5.1

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/mysql-5.1/+bug/688541/+subscriptions