News for hackers. Stuff that matters.
Re:GitLab Database Incident – Live Report by YorickPeterse on Tuesday January 31, 2017 @08:03PM (#13537132)
Nope, that would be me.
Re:GitLab Database Incident – Live Report by bigiain on Tuesday January 31, 2017 @08:10PM (#13537185)
Tough night dude. I'll buy you a drink or three if you're ever in Sydney...
Re:GitLab Database Incident – Live Report by refulgentis on Tuesday January 31, 2017 @08:05PM (#13537151)
Alas, poor Yorick!
Re:GitLab Database Incident – Live Report by sytse on Tuesday January 31, 2017 @08:18PM (#13537257)
Sorry for the rough night Yorick. This could happen to all of us but of course it happens to the person that is working the hardest. <3
Re:GitLab Database Incident – Live Report by kfrzcode on Tuesday January 31, 2017 @08:37PM (#13537359)
I'm going to add to your message box unnecessarily, but I want to say I love GitLab and it's a shining example of a transparent company. I still have ambitions to work there someday, and this event is hopefully a net gain in the end, in that everyone here and there learns about backups.
Re:GitLab Database Incident – Live Report by AlexB138 on Tuesday January 31, 2017 @08:07PM (#13537159)
Thanks for the transparency. Doesn't always feel good to have missteps aired in public, but it makes us all a little better as a community to be clear about where mistakes can be made.
Re:GitLab Database Incident – Live Report by jdhendrickson on Tuesday January 31, 2017 @09:19PM (#13537609)
I've been there myself, it was at the start of my career and almost ended it. I know how incredibly emotional this kind of thing can be. Just understand you aren't the first, you wont be the last and shit happens. If you are ever in Philly I'll buy you a beer if you drink, and a dinner if you don't.
Re:GitLab Database Incident – Live Report by reis4g on Wednesday February 01, 2017 @04:23AM (#13539479)
Was just trying to push earlier today and found out about the issue. Sorry man! Drinks on me in Montevideo, Uruguay. This stuff happens, more than most of us are willing to accept so, here is for your transparency and you know, fix it, learn it and on you go!
Re:GitLab Database Incident – Live Report by toomuchtodo on Tuesday January 31, 2017 @09:07PM (#13537540)
Infrastructure ops here. Happens to the best of us. Just have to get back on the horse.
Re:GitLab Database Incident – Live Report by haldean on Tuesday January 31, 2017 @09:41PM (#13537714)
All I could think as I read the postmortem was "I hope YP is doing okay". We've all been there, and I hope you're doing okay!
Re:GitLab Database Incident – Live Report by ben_jones on Tuesday January 31, 2017 @08:23PM (#13537289)
I'll pour one out for you next time I go out.
Re:GitLab Database Incident – Live Report by gadders on Tuesday January 31, 2017 @08:50PM (#13537453)
Unlucky mate. Even monkeys fall out of trees. Good luck with the fixing.
Re:GitLab Database Incident – Live Report by chx on Wednesday February 01, 2017 @05:25AM (#13539659)
Dude, we've all been there. You are neither the first nor the last. It's never a single person. One day the technical side of our little startup collectively went to a conference (back then it was only two engineers plus a technical leader) while our server DoS'd itself by broken mail processing... we had a rough night in a Belgium hotel figuring out who attacked the site to realize it was ourselves. The 10k block of missing image IDs always stood as a reference not to leave even a low traffic site unmonitored. It happens.
Re:GitLab Database Incident – Live Report by cheshire137 on Tuesday January 31, 2017 @10:59PM (#13538116)
Hey man, don't beat yourself up over this. It's shitty but you found some flaws in the process, in the setup, and y'all can make things better because of that.
Re:GitLab Database Incident – Live Report by petetnt on Wednesday February 01, 2017 @05:49AM (#13539750)
Been there, done that (like most of us that deal with any kinds of infra). Hang in there and best of luck on the recovery efforts!
Re:GitLab Database Incident – Live Report by q3k on Tuesday January 31, 2017 @09:30PM (#13537662)
Yorick! Thank you for the transparency, I know how tough incidents like these can be. Stop the bleeding, figure out how to handle this better in the future, but most of all, take care!
Re:GitLab Database Incident – Live Report by duncan_bayne on Wednesday February 01, 2017 @07:24AM (#13540147)
Hey - thanks for GitLab, and my condolences on your (data) loss.
Re:GitLab Database Incident – Live Report by Raticide on Tuesday January 31, 2017 @10:15PM (#13537891)
It happens to everyone eventually, so don't feel too bad. But at least now you'll have a much more robust backup process.
Re:GitLab Database Incident – Live Report by nstj on Wednesday February 01, 2017 @04:20AM (#13539462)
We've all been there! Feel better man, it could be a lot worse :)
Re:GitLab Database Incident – Live Report by makoz on Tuesday January 31, 2017 @10:31PM (#13537981)
Go sleep man, shit happens
Re:GitLab Database Incident – Live Report by m_fam_wa_k on Tuesday January 31, 2017 @09:34PM (#13537673)
Always welcome for a drink when you are in Osaka/Tokyo.
Re:GitLab Database Incident – Live Report by antocv on Wednesday February 01, 2017 @08:51AM (#13540659)
You have drinks of your choice in Stockholm too. Man hug.
Re:GitLab Database Incident – Live Report by lazyant on Wednesday February 01, 2017 @09:49AM (#13541093)
hugs