News for hackers. Stuff that matters.
Re:Data Loss at GitLab by neals on Wednesday February 01, 2017 @02:40PM (#13544216)
I don't get to work on databases this size and today has been an incredible lesson and a journey. I've been reading all the comments and blogs, watching the stream and Googling what I didn't know or understand.

I feel like the next step for me is scaling my business so that we have an actual usage for my newly found interests :)

Re:Data Loss at GitLab by sulam on Wednesday February 01, 2017 @03:11PM (#13544598)
Then you end up having deep expertise in a topic that's only important for larger companies. Sometimes this works out well, sometimes it leaves you a little stuck. :)
Re:Data Loss at GitLab by stusmall on Wednesday February 01, 2017 @03:50PM (#13544999)
Meh. Just learn, learn, learn. If it isn't completely applicable to your life that's okay. Sometimes there are pearls of wisdom in best practices in field completely unrelated to your own. Learning something new is always a good thing.
Re:Data Loss at GitLab by haggy on Thursday February 02, 2017 @07:57AM (#13550178)
THIS ^^^ So much this. I can't tell you how many times I picked up a book or paper thinking "There's no way i'm going to get anything new out of this". As I start reading, I start to find little tidbits of information that make me think in ways I didn't before.
Re:Data Loss at GitLab by skywhopper on Wednesday February 01, 2017 @03:50PM (#13544990)
Oh, there's plenty of demand for skilled DBAs with deep expertise with handling large volumes of data, even at small-to-medium-sized companies. GitLab being one of those companies! In fact, demand for this sort of thing is likely to increase. If it interests you, dig in. You'll never stop learning.
Re:Data Loss at GitLab by hyperpape on Wednesday February 01, 2017 @03:52PM (#13545014)
I'm not sure if it's really company size, though there's probably a correlation. My company is only marginally bigger than GitLab in terms of employees (just recently 200+), and we have several dozen databases bigger than this.
Re:Data Loss at GitLab by wkd on Wednesday February 01, 2017 @04:41PM (#13545556)
This, it depends more on the product and the requirements. I've worked in companies that are just a few employees and are running databases larger than this. I've also worked for a company with ~100 employees whose only 'database' is an in memory cache and a few kilobytes big textfile
Re:Data Loss at GitLab by jjirsa on Thursday February 02, 2017 @02:52AM (#13548876)
Running databases at scale is great - it's what I do for a living, and I wouldn't want to do anything else

But please realize that this is a horrible example. Almost everything done was wrong, technical choices, processes, everything - please don't use it as a positive example.

Re:Data Loss at GitLab by anw on Wednesday February 01, 2017 @04:09PM (#13545216)
For those wondering about the live Gitlab stream, you can watch them work here:

https://www.youtube.com/watch?v=nc0hPGerSd4

Re:Data Loss at GitLab by cs02rm0 on Thursday February 02, 2017 @04:50AM (#13549337)
That is remarkably transparent. Commendable.
Re:Data Loss at GitLab by throwaway7767 on Thursday February 02, 2017 @06:36AM (#13549785)
It sounds like a terrible working environment for the sysadmins. When shit is broke, you focus on fixing it. Being on camera is distracting, and setting up the livestream probably takes a bit of time that could be spent on actually fixing the problem.
Re:Data Loss at GitLab by jobvandervoort on Thursday February 02, 2017 @07:01AM (#13549908)
We had the call live with all engineers and asked if everyone was okay with streaming this.

Someone not involved with helping fixing the problem set up the stream from their home, while we continued work as normal.

I think the overall spirit was that it was comfortable to do it like this. Note that no one was required to work like this and we'd happily stopped streaming if anyone would have any problems with it.

Re:Data Loss at GitLab by YorickPeterse on Wednesday February 01, 2017 @07:23PM (#13546867)
It's both fun and incredibly frustrating at times.