News for hackers. Stuff that matters.
Re:Summary of the Amazon S3 Service Disruption by gigatexal on Thursday March 02, 2017 @01:01PM (#13776465)
Yeah as soon as I read this I felt bad for the employee. I remember writing an update statement without a where clause and having to restore the table from backup. But that was at a company not as advanced as Amazon. Fat fingering a key like that is just crazy (but comforting that even at Amazon it happens) and I'm sure they've fixed that from happening again.
Re:Summary of the Amazon S3 Service Disruption by duskwuff on Thursday March 02, 2017 @01:37PM (#13776839)
FWIW: Setting "safe-updates=1" in ~/.my.cnf will require UPDATE and DELETE statements in the client to have a WHERE clause which references a key. It's not perfect protection, but it will save you from a lot of mistakes.
Re:Summary of the Amazon S3 Service Disruption by amichal on Thursday March 02, 2017 @04:14PM (#13778281)
That's awesome,

My worst DELETE fail however was:

  DELETE * FROM table WHERE [long condition that resolves to true for all records]
Now i write SELECT or SELECT COUNT(*) over and over again until i see the data i expect and then change it to a DELETE/UPDATE.

It's not my personal habit but some folks I know turn off auto commit and BEGIN a transaction every time they enter an interactive SQL sessions. They then default to ROLLBACK at least once before COMMITing them.

That and having a user with read-only permissions or a read replica

Re:Summary of the Amazon S3 Service Disruption by tacon on Thursday March 02, 2017 @04:31PM (#13778432)
Is there a collection of data safety tips like this somewhere? I never knew this existed. What else am I missing?
Re:Summary of the Amazon S3 Service Disruption by gigatexal on Thursday March 02, 2017 @01:39PM (#13776854)
hmm, that's kinda cool. I'm in a MS shop and I don't know if SMSS has the same feature. My manager just looked at me and said "welp, go restore the table and be more careful next time." I was a new DBA at the time, still, kinda new.
Re:Summary of the Amazon S3 Service Disruption by jsudhams on Thursday March 02, 2017 @10:26PM (#13780523)
USE BEGIN TRANS as mentioned above
Re:Summary of the Amazon S3 Service Disruption by vocatus_gate on Thursday March 02, 2017 @02:28PM (#13777353)
I once brought down our entire production XenServer cluster group by issuing a "shutdown now" in the wrong SSH window. Needless to say it was a bad feeling watching Nagios go crazy and realizing what had just happened.
Re:Summary of the Amazon S3 Service Disruption by Piskvorrr on Thursday March 02, 2017 @05:14PM (#13778773)

    root@baz # shutdown now
    W: molly-guard: SSH session detected!
    Please type in hostname of the machine to shutdown: foo
    Good thing I asked; I won't shutdown baz ...
Surprising to see such a simple protection neglected.
Re:Summary of the Amazon S3 Service Disruption by twunde on Thursday March 02, 2017 @07:15PM (#13779654)
I don't know how well-known molly-guard is, but I've never heard of it before. Definitely enabling it on my servers next week.
Re:Summary of the Amazon S3 Service Disruption by Piskvorrr on Friday March 03, 2017 @02:58PM (#13785795)
Interesting. Up until now I've considered it well-known to the point of ubiquity :)
Re:Summary of the Amazon S3 Service Disruption by gigatexal on Thursday March 02, 2017 @03:22PM (#13777812)
Oh crap! Yeah I bet you were pretty panicked. My update statement destroyed data that my team used all the time so I was worried I'd get fired. Luckily that wasn't the case.
Re:Summary of the Amazon S3 Service Disruption by saalweachter on Thursday March 02, 2017 @02:14PM (#13777227)
Fat fingers are just nature's way of making sure you test your back-up & restore procedures periodically :-)
Re:Summary of the Amazon S3 Service Disruption by wakkaflokka on Thursday March 02, 2017 @08:34PM (#13780071)
When I first started using Linux and wanted to do some housecleaning, I did "rm -r (asterix) in a folder. Cleaned up everything, no prob. Then went to some more folders, hit the up arrow on my keyboard fast to get to a command I had used before. Hit 'enter' before my brain realized I had landed on rm -r (asterix) and not the right command. Never used that command again.