News for hackers. Stuff that matters.
Re:Paxos vs. Raft: Have we reached consensus on distributed ... by goodpoint on Friday July 16, 2021 @06:05PM (#27861761)
Most importantly: leader election should be used only as last resort.

Leaderless systems with eventual consistency are often possible and more robust.

Re:Paxos vs. Raft: Have we reached consensus on distributed ... by titogeo on Saturday July 17, 2021 @01:27AM (#27863709)
Good point, Consensus at high data scales is not achievable. We are building a cloud-native log analytics solution at logiq.ai. Our initial design had RAFT at the core. Though it is relatively easy to implement when we started hitting high data rates (50+ Gb per hour), the RAFT APIs started to slow down the entire system. We removed it and made each pod stateless, which allowed us to scale exponentially. Now we can go from 1 Gb per hour to hundreds with a single `kubectl scale sts` command. Eventual consistency is the way forward for applications that generates high volumes of data.

Of course, some applications do require Consensus; for those, I would start with RAFT as a starting point.

Re:Paxos vs. Raft: Have we reached consensus on distributed ... by asdfasgasdgasdg on Friday July 16, 2021 @10:27PM (#27863154)
I don't agree with this. Eventually consistent was majorly hyped a while ago, but these days I don't think that there's widespread consensus on this issue. The complexity of getting an eventually consistent system working correctly is often much higher than in a system with a leader.
Re:Paxos vs. Raft: Have we reached consensus on distributed ... by extrapickles on Saturday July 17, 2021 @12:39AM (#27863561)
Most large real-time systems (or near enough) must be eventually consistent due to speed of light delays between all nodes of the system.

Multiplayer video games have big problems with light speed. A user could be 100ms away from the server, and another 100ms in the other direction, so anything one user does, the other will see 200ms later. If the game wasn't eventually consistent, it would need to be a turn based game as 5 actions per second is too slow as most real-time games run 60-100 actions per second.

A bank account is also eventually consistent as transactions need several days to clear, causing the exact balance to be unknown. This is why a bank establishes an "available" balance as that is the most pessimistic estimation of an accounts balance.

Like with all things, designing a system as eventually consistent or as a leader-type really depends on the application, the team that is going to build it and the resources available to support it.

Re:Paxos vs. Raft: Have we reached consensus on distributed ... by saati on Saturday July 17, 2021 @05:50AM (#27864687)
Where are bank transactions so slow? In the EU they take a few seconds now.
Re:Paxos vs. Raft: Have we reached consensus on distributed ... by extrapickles on Saturday July 17, 2021 @09:31AM (#27865637)
Its so mistakes can be corrected. The transaction will show up in the account in a few seconds, but it takes a few days to be final.
Re:Paxos vs. Raft: Have we reached consensus on distributed ... by funny_falcon on Saturday July 17, 2021 @02:59AM (#27863996)
Consensus on eventual consistency is eventual.