News for hackers. Stuff that matters.
Re:What we learned after I deleted the main production datab... by SoftTalker on Monday September 19, 2022 @05:36PM (#32904364)
> In the fifteen minutes I had before the next meeting, I quickly joined with one of my senior members to quickly access the live environment and perform the query.

Don't do stuff in a rush like this. That's when I almost always make my worst mistakes. If there is a "business urgency" then cancel or get excused from the upcoming meeting so you can focus and work without that additional pressure. If the meeting is urgent, then do the other task afterwards.

Re:What we learned after I deleted the main production datab... by cranekam on Tuesday September 20, 2022 @05:25AM (#32909511)
And then, after you're done hand-rolling queries with Postman, immediately start planning actual, safe, tools to do this kind of thing when the need crops up.
Re:What we learned after I deleted the main production datab... by fijiaarone on Tuesday September 20, 2022 @11:45PM (#32920634)
This story is actually about a really smart person making a stupid mistake and humbly admitting it.

Hugo has some other interesting takes on his blog.

I don’t know him. But just because you haven’t done the exact same thing — or never been is a position of responsibility where your mistakes can affect many other people, doesn’t mean that everyone doesn’t make mistakes.

It just means that not everyone can learn from others mistakes.

Re:What we learned after I deleted the main production datab... by cranekam on Saturday October 01, 2022 @07:59AM (#33045945)
(very late reply, only just saw this)

I'm not claiming to have never made a similar mistake. I have messed up in production a lot. But I would definitely recognise from this episode that the lack of good tooling led to risky, rushed, manual processes that can easily go wrong.

Re:What we learned after I deleted the main production datab... by racl101 on Monday September 19, 2022 @06:31PM (#32905030)
Now this meeting will be get many more urgent meetings.
Re:What we learned after I deleted the main production datab... by Cthulhu_ on Tuesday September 20, 2022 @09:51AM (#32911656)
Which you don't have to attend, because you're the lead IT guy whose job is to solve the crisis, not reassure the meeting people.

I work at an energy company at the moment, the servers are overloaded because more old-fashioned IT and there's an energy crisis ongoing tripling people's monthly expenses and putting them into poverty. Everyone's panicking, but I hope they're not trying to force quick fixes or skipping due process. We're still doing code reviews and going through our regular deployment processes.