[Bug 620441] Re: MySQL upstart stop job does not cleanly shutdown mysql
clint at fewbar.com
Sat Nov 20 20:38:40 GMT 2010
** Description changed:
Release: Ubuntu 10.04 LTS
Package Version: 5.1.41-3ubuntu12.6
Stopping a MySQL server can take some time, especially on a busy
database. If mysqld gets killed while having open connections, then
after the next start you will likely have a lot of tables corrupted.
The new upstart mechanism for starting/stopping mysql unfortunately behaves as follows on a "service mysql stop":
1. First it will send a TERM signal to mysqld which is perfectly fine as mysqld will do the same as if you would issue a mysqladmin shutdown command
2. If after 5 seconds the process is still not terminated, a KILL signal will be send. This is really a very big issue since on busy servers lots of myisam tables will be corrupted afterwards
Send the term signal and wait for a certain amount of time (at least 1 minute). If still not stopped, then simply give an error about this to the user
+ == SRU REPORT ==
+ === IMPACT ===
+ This has very little impact on a normal running system. It may cause
+ shutdown to take up to 5 minutes, but that is an acceptable trade off as
+ recovering a system with crashed tables can take much, much longer.
+ === DEV FIX ===
+ Raising kill timeout to 300 seconds seems a good trade off between
+ shutdown potentially taking forever, and trying not to corrupt tables.
+ === PATCH ===
+ See merge proposal
+ === TEST CASE: ===
+ This is a race condition, so it is hard to reproduce reliably. However,
+ if you:
+ 1. create a very large innodb table
+ 2. import a lot of data (inserting more MB of data than your drives can write in 5 seconds should work)
+ 3. run a very long select on the data in one thread (SELECT * FROM table WHERE column like '%X%';)
+ 4. DELETE that data in another thread
+ 5. issue 'stop mysql' immediately after the delete returns.
+ 6. check to see that mysqld was sent SIGKILL in /var/log/daemon.log
+ 7. start mysql -- at this point InnoDB recovery will need to be run
+ Once the fix is applied, the SIGKILL should never be sent.
+ === REGRESSION POTENTIAL ===
+ The only regression potential is mentioned in the IMPACT section, that
+ the shutdown of a server or the mysql service may take up to 5 minutes.
MySQL upstart stop job does not cleanly shutdown mysql
You received this bug notification because you are a member of Ubuntu
Server Team, which is subscribed to mysql-dfsg-5.1 in ubuntu.
More information about the Ubuntu-server-bugs