Exactly-once Semantics: How Kafka Does it

Re:Exactly-once Semantics: How Kafka Does it by zekrioca on Friday June 30, 2017 @01:36PM (#14672314)

Probably these new over-layers of abstractions are needed for some use-cases in for specialized and more performant networks. Otherwise, the simple fact of using TCP would be enough.

But indeed, it is interesting to see the computing cycle reimplementations spinning over and over again.

[ Parent | on HN ]

Re:Exactly-once Semantics: How Kafka Does it by phamilton on Friday June 30, 2017 @05:17PM (#14673659)

Yeah, my point wasn't "TCP does this, that's all we need". It was just an observation about how we apply principles at a higher level.

[ Parent | on HN ]

Re:Exactly-once Semantics: How Kafka Does it by nehanarkhede on Friday June 30, 2017 @01:50PM (#14672426)

Re: TCP, it only maintains sequencing guarantees over the lifetime of a single connection. Obviously this is too weak of a guarantee for Kafka as leaders can change in a cluster. We've built idempotence in a way that makes the sequence number and producer ID part of the Kafka log. So it can provide idempotence even if brokers fail and new connections are established between the producer and broker

[ Parent | on HN ]