News for hackers. Stuff that matters.
Re:Why we rolled our own consensus algorithm by zzzcpan on Thursday November 15, 2018 @01:36PM (#18461923)
If you have enough background in distributed systems to choose a consensus algorithm you are definitely capable of implementing one yourself and even invent your own. It's not a bad idea either, most existing consensus-based systems are absolute garbage. People just hugely overestimate how much those algorithms matter for production systems.

EDIT: Note, I'm not talking about blockchains and byzantine fault tolerance, but in context of Paxos/Raft.

Re:Why we rolled our own consensus algorithm by eloff on Thursday November 15, 2018 @01:46PM (#18462030)
That's like saying if you have some crypto background you can implement your own crypto library. It's TERRIBLE advice.

Read the Google paper on their chubby distributed locking system using paxos. They had a hard time getting it right - unless you think you're so much better than them. IIRC Leslie Lamport works (worked?) for them.

Use an existing, well used, well scrutinized implementation of a known algorithm.

Re:Why we rolled our own consensus algorithm by kjeetgill on Thursday November 15, 2018 @01:52PM (#18462070)
Are you sure? I thought Lamport has been at Microsoft Research since forever.
Re:Why we rolled our own consensus algorithm by therein on Thursday November 15, 2018 @03:08PM (#18462824)
You're correct. Lamport has joined MSR in 2001 right after publishing "Time, Clocks, and the Ordering of Events in a Distributed System" and has been there ever since.

I am not sure if he is still at the MV office but he was one of the reasons why MSR kept a presence in MV for a while.

Re:Why we rolled our own consensus algorithm by lou1306 on Thursday November 15, 2018 @08:03PM (#18465222)
Actually "Time, Clocks..." is a paper from the late 70s. Maybe you were thinking of the Paxos paper, "The Part-Time Parliament" (1998)?
Re:Why we rolled our own consensus algorithm by therein on Thursday November 15, 2018 @08:56PM (#18465606)
Yeah, copy-pasted the wrong title and by the time I noticed, it was too late to edit. :)
Re:Why we rolled our own consensus algorithm by eloff on Thursday November 15, 2018 @04:20PM (#18463462)
Ah the vagaries of the human memory.
Re:Why we rolled our own consensus algorithm by apersona on Thursday November 15, 2018 @04:03PM (#18463287)
If you're a normal software engineer, I agree, you shouldn't implement it yourself.

But if you're a researcher/had a CS research background, I don't see why not. By definition research results in novel approaches (ideally better) than the current ones. You have to be willing to challenge the consensus.

PhD students who can prove that their algs work should work on it, like the results of this paper: https://raft.github.io

Re:Why we rolled our own consensus algorithm by dlubarov on Thursday November 15, 2018 @01:54PM (#18462090)
> Use an existing, well used, well scrutinized implementation of a known algorithm.

Keep in mind that this is BFT consensus. Until recently, it was a niche area of research with very little real world usage. Even if an existing algorithm like PBFT met their needs, there are no well tested PBFT libraries.

Re:Why we rolled our own consensus algorithm by drfuchs on Thursday November 15, 2018 @02:50PM (#18462650)
No, Lamport does not now, and never has, worked at Google.