News for hackers. Stuff that matters.
Re:Why Was Apache Kafka Created? by Joeri on Saturday August 23, 2025 @04:26PM (#44998863)
Kafka isn’t a queue, it’s a distributed log. A partitioned topic can take very large volumes of message writes, persist them indefinitely, deliver them to any subscriber in-order and at-least-once (even for subscribers added after the message was published), and do all of that distributed and HA.

If you need all those things, there just are not a lot of options.

Re:Why Was Apache Kafka Created? by holografix on Saturday August 23, 2025 @10:29PM (#45000785)
Finally a valuable answer thank you
Re:Why Was Apache Kafka Created? by diarrhea on Sunday August 24, 2025 @04:13AM (#45002352)
Perhaps the best terse summary of Kafka I have come across yet.
Re:Why Was Apache Kafka Created? by edem on Sunday August 24, 2025 @04:52AM (#45002571)
What do you think about Temporal?
Re:Why Was Apache Kafka Created? by cyberpunk on Sunday August 24, 2025 @01:32PM (#45006047)
Okay for small numbers of high value jobs (e.g uber trips or food deliveries etc), prohibitively expensive for anything you need even a few k/sec of.
Re:Why Was Apache Kafka Created? by HarHarVeryFunny on Sunday August 24, 2025 @08:01AM (#45003574)
Why do you say log rather than just publish and subscribe?
Re:Why Was Apache Kafka Created? by exitb on Sunday August 24, 2025 @08:15AM (#45003663)
Clients don’t have to subscribe to latest messages, but rather can request any available offset range.
Re:Why Was Apache Kafka Created? by halifaxbeard on Sunday August 24, 2025 @10:23AM (#45004475)
the log stays on kafka for replay until your per log retention settings delete it
Re:Why Was Apache Kafka Created? by HarHarVeryFunny on Sunday August 24, 2025 @11:16AM (#45004859)
The way people choose to use feedback on HN never fails to suprise me - we've got a generally intelligent user base here, but the most common type of feedback voting isn't because something is wrong but rather a childish "I don't like it - I want to suppress this comment".

In this case it's something different - this was an honest question, and received two useful replies, so why downvote?! The mental model of people using Kafka is useful to know - in this case the published data being more log-like than stream-like since it's retained per a TTL policy, with each "subscriber" having their own controllable read index.

Re:Why Was Apache Kafka Created? by MYEUHD on Sunday August 24, 2025 @12:11PM (#45005363)
> Please don't comment about the voting on comments. It never does any good, and it makes boring reading.

https://news.ycombinator.com/newsguidelines.html#comments

Re:Why Was Apache Kafka Created? by HarHarVeryFunny on Sunday August 24, 2025 @02:12PM (#45006356)
Same goes for metacomments about voting.
Re:Why Was Apache Kafka Created? by mannycalavera42 on Sunday August 24, 2025 @08:54AM (#45003907)
HERO
Re:Why Was Apache Kafka Created? by iamcreasy on Friday August 29, 2025 @02:47PM (#45067906)
Why the delineation? As long as you have a queue - you can publishing whatever you want in it, right? Be it log or something else.