News for hackers. Stuff that matters.
Re:You Cannot Have Exactly-Once Delivery by jongjong on Wednesday March 01, 2023 @06:44PM (#34989991)
This article is incorrect.

> The way we achieve exactly-once delivery in practice is by faking it. Either the messages themselves should be idempotent, meaning they can be applied more than once without adverse effects, or we remove the need for idempotency through deduplication.

The author is wildly overestimating the complexity of the solution by ignoring the existence of a very simple but critical concept: UUIDs

If the client generates and assigns a UUID to each message it sends, the receiver can easily check if a specific message was already received before by comparing it against previously received UUIDs and can discard duplicates.

The title of the article is misleading because exactly-once delivery is in fact possible by deduplication. The deduplication can occur automatically (without having to know any context about the message beyond its UUID); it can occur BEFORE the delivery of the message to the consumer... So the title and entire premise of the article is incorrect.

What the author should really be saying is that more than one copy of the same message may occasionally pass through the network due to failures... But they are misleading in suggesting that it's not possible to deliver exactly one copy or that deduplication is a herculean task when in fact it's trivial.

Re:You Cannot Have Exactly-Once Delivery by gmcquistin on Wednesday March 01, 2023 @06:57PM (#34990097)
> the receiver can easily check if a specific message was already received

It seems like you're conflating exactly-once delivery with exactly-once processing. Your proposed solution is only necessary _because_ we can't have exactly-once delivery as TFA states.

Re:You Cannot Have Exactly-Once Delivery by jongjong on Wednesday March 01, 2023 @07:36PM (#34990417)
I don't agree with that contrived definition of delivery. As a developer, what I care about is delivery of the message to my function/handler. With that definition, you can definitely have exactly-once delivery. The developer's function handler is guaranteed to only be passed each message exactly once (assuming that the network doesn't go down permanently).
Re:You Cannot Have Exactly-Once Delivery by mankyd on Wednesday March 01, 2023 @07:00PM (#34990128)
>> we remove the need for idempotency through deduplication.

> If the client generates and assigns a UUID to each message it sends, the receiver can easily check if a specific message was already received before by comparing it against previously received UUIDs and can discard duplicates.

You quoted the part of the article that suggests exactly your solution: deduplication.

Re:You Cannot Have Exactly-Once Delivery by jongjong on Wednesday March 01, 2023 @07:07PM (#34990178)
Yes but the author suggests that it's difficult to perform or requires constructing messages in a complicated way which is not the case. Generating and attaching a UUID to a message is very simple, so is checking whether or not a UUID has already been encountered on the receiver side.

The author's definition of 'message delivery' is incorrect and so is the title which is clickbait.

Re:You Cannot Have Exactly-Once Delivery by greggyb on Wednesday March 01, 2023 @07:20PM (#34990284)
Perhaps adding a UUID is too much overhead? If you have a very high throughput system with messages of less than 128 bits, then the addition of a UUID to every message would effectively cut throughput in half.

I do not have such a use case in mind, but the description above does not sound terribly unrealistic to me.

Re:You Cannot Have Exactly-Once Delivery by datadata on Wednesday March 01, 2023 @07:28PM (#34990368)
Also in order to track UUIDs that have been received, you would either need an infinite amount of memory, or have the system bounded in terms of how many messages out of order a message be delivered and still be processed correctly.
Re:You Cannot Have Exactly-Once Delivery by jongjong on Wednesday March 01, 2023 @07:57PM (#34990564)
You can use a database. Practically, most systems will keep a record of each message on disk in any case (at least they will have some logs) so it's not that outrageous. Safe to assume that the receiver can discard non-authenticated messages to avoid spam.
Re:You Cannot Have Exactly-Once Delivery by datadata on Saturday March 04, 2023 @09:45AM (#35021064)
It depends entirely on volume of messages vs available memory (ram or disk) how outrageous it is
Re:You Cannot Have Exactly-Once Delivery by birdyrooster on Wednesday March 01, 2023 @06:50PM (#34990037)
If all of your agents are seeing the UUID for the first time, they cannot deduplicate amongst themselves without quorum or shared state which has race conditions itself. It’s non trivial.
Re:You Cannot Have Exactly-Once Delivery by jongjong on Wednesday March 01, 2023 @07:18PM (#34990270)
There's no need to. Different agents will have different copies of the same message, that's a given. If the sender ensures that a UUID is attached to a single specific message and never re-used (though they may retry to send the exact same message multiple times), then there is no risk that multiple receivers would have different versions of a message and therefore consensus is guaranteed without even having to talk to each other.

If you assume that the sender cannot be trusted to not reuse a single UUID for multiple messages (e.g. to trick the receiver), you can still work around that limitation by computing and comparing message hashes (e.g. sha256) on the receiver side... Don't even need UUIDs. Every time you receive a message, you can hash it and check if you're received a message with this hash before (storing the hash of each message as it is received). You can use the hash in place of a UUID though in this case you probably need to add some index to each message to ensure that each hash is unique over time (since a message with the exact same payload will be counted as the same message, even if broadcast a long time apart).

Re:You Cannot Have Exactly-Once Delivery by preseinger on Wednesday March 01, 2023 @07:53PM (#34990532)
exactly once delivery is not satisfied if the same message is delivered to the recipient multiple times
Re:You Cannot Have Exactly-Once Delivery by jongjong on Wednesday March 01, 2023 @08:06PM (#34990631)
It depends on your definition of 'recipient' and 'delivery'. If you assume that the receiver is a specific database instance (or shard) or a specific data store (on the receiver side), then it's totally possible to have exactly once delivery in the sense that each message sent would end up being inserted into the database/data store exactly once without duplicates.
Re:You Cannot Have Exactly-Once Delivery by preseinger on Wednesday March 01, 2023 @08:25PM (#34990756)
but you can't make those assumptions, in general
Re:You Cannot Have Exactly-Once Delivery by EdwardDiego on Wednesday March 01, 2023 @07:14PM (#34990233)
Yeah, non trivial is correct.

A common Kafka approach is to partition by key, so that a given UUID will only be placed on one partition, and we're guaranteed that any further messages with that key will also be placed on that partition, so handled by the same consumer.

Then create a change-log topic that's co-partitioned by key with the input topic. And then funk around with partition assignment strategies so if consumer X is assigned partition 10 of the input topic, then it's also assigned partition 10 of the change-log topic.

Then add RocksDB as a fast state store on a persistent volume claim, as restoring state fresh from your change-log topic turns out to take about 7 minutes.

And then realise you've just reimplemented bits of Kafka Streams poorly.

Re:You Cannot Have Exactly-Once Delivery by vlovich123 on Wednesday March 01, 2023 @06:54PM (#34990072)
But not impossible.
Re:You Cannot Have Exactly-Once Delivery by birdyrooster on Wednesday March 01, 2023 @07:14PM (#34990237)
Depending on your ability to share state between these agents it could very well be impossible. It depends on other factors.
Re:You Cannot Have Exactly-Once Delivery by prng2021 on Wednesday March 01, 2023 @08:15PM (#34990691)
You’re completely missing the point. Many multi master databases exist. Network partitions happen. Bugs in multi master databases during network partitions have happened and can happen. This is like distributed systems 101.

So given those facts, your design of using UUIDs is absolutely not full proof if the database where you stored your UUID is not in a consistent state.

Here’s some more useful info for your perusal (https://ucare.cs.uchicago.edu/pdf/socc14-cbs.pdf)

“ Data consistency means that all nodes or replicas agree on the same value of a data (or eventually agree in the context of eventual consistency). In reality, there are several cases (5%) where data consistency is violated and users get stale data or the system’s behavior becomes erratic. The root causes mainly come from logic bugs in operational protocols (43%), data races (29%) and failure handling problems (10%).1 Be- low we expand these problems.”

Re:You Cannot Have Exactly-Once Delivery by KptMarchewa on Wednesday March 01, 2023 @08:33PM (#34990797)
Your database does not have to be networked, it can be on-disk KV store like RocksDB, which is what things like Flink state backend use.