News for hackers. Stuff that matters.
Re:When should I not kill -9 a process? by falcolas on Saturday May 24, 2014 @10:29AM (#7793389)
You, and the OP, are likely mistaken about the cause of the corruption. MySQL, running InnoDB, is ACID compliant. InnoDB takes that durability seriously, and even by hand-tuning the performance factors, it's very hard to put InnoDB in a state where a simple process death, even during shutdown, will corrupt the files on disk.

If the database truly corrupted only due to improper shutdown, it was because the double write buffers were disabled on a non-atomic FS in which case you're intentionally risking DB corruption. More likely, there was bit-rot which was undetected during the normal running state of MySQL, and recovery couldn't get around it.

On the anecdote side, I've yet to have MySQL corrupt a database from a kill -9, and as I do a lot of failover development and testing, I'm issuing kill -9 to running databases frequently.

Re:When should I not kill -9 a process? by kator on Saturday May 24, 2014 @11:00AM (#7793438)
This was years ago and trust me it was corrupt. I think there may have been some issues with InnoDB in those days that made it a bit fragile in that situation.
Re:When should I not kill -9 a process? by falcolas on Saturday May 24, 2014 @11:10AM (#7793458)
My apologies if it sounded like I was doubting that there was corruption - I was not. I was doubting that merely sending kill -9 to the process was the sole cause of the corruption. More than likely it uncovered the corruption due to having to do recovery against corrupted data.
Re:When should I not kill -9 a process? by kator on Saturday May 24, 2014 @01:00PM (#7793761)
In the postmortem it seemed that by issuing the shutdown but not waiting for it to complete the database ended up in a strange state that we could never fully recover from.

I guess the moral here is tread lightly and take the time to be informed before you just smack something with a club.

Re:When should I not kill -9 a process? by falcolas on Saturday May 24, 2014 @05:30PM (#7794466)
++ to the moral of the story.
Re:When should I not kill -9 a process? by yuubi on Saturday May 24, 2014 @01:18PM (#7793811)
> double write buffers were disabled on a non-atomic FS

Surely just sending SIGKILL will allow the writes already issued to complete, regardless of the FS type/options? Nobody kicked the plug out of the wall.

Re:When should I not kill -9 a process? by falcolas on Saturday May 24, 2014 @04:53PM (#7794390)
Double write buffers help to validate that the pages were written to disk correctly. Otherwise, there's no way to tell if a complete page was written, since writing 16k pages to disk is not an atomic operation.

InnoDB writes to the doublewrite buffer (an internal tablespace on disk), then the record in place. If the two do not match on recovery, the record is considered not written, and rebuilt from the transaction log.

If the doublewrite buffer is disabled, then InnoDB has no way of telling if the write to the page on disk was started, completed, or partially completed. This causes corruption.