Re:Why Was Apache Kafka Created?
by kachapopopow on Sunday August 24, 2025 @04:44AM
(#45002538)
I do not recommend Redis (janky implementation, subscribers drop randomly, java libraries are a crime against humanity) or RabbitMQ (memory issues), ZMQ is not a messaging queue, named pipes are not reliable and what the hell is SQS.
[ Parent
| on HN ]
How have you had so many issues with Redis. We used it at a previous place it was basically bullet proof. That being said we didn’t use Java but Python. Idk.
Re:Why Was Apache Kafka Created?
by kachapopopow on Sunday August 24, 2025 @12:03PM
(#45005288)
lettuce - doesn't reconnect properly during redis restarts (1/10 chance)
jedis - subscriptions drop and stop receiving for no reason
however, my latest wrapper for jedis does seem to be holding up and haven't had too many issues, but I have a very robust checking for dropped connections.
SQS is simple queueing service in AWS. It's ok, guarantees at least one time delivery, but I am not sure how useful it is for large volumes of messages (by this I just mean that we use it for low volume messages and I don't have experience when using larger volumes).
Re:Why Was Apache Kafka Created?
by stickfigure on Sunday August 24, 2025 @05:24PM
(#45007922)
SQS is fantastic at exceptionally high total volumes of messages - you probably can't saturate it. But it's not great for streaming a list of ordered messages. SQS has a FIFO mode but performance will never be what you can get out of Kafka.
Also, SQS isn't pub/sub. Kafka and SQS really have very different use cases.