Apache Heron: A realtime, distributed, fault-tolerant stream processing engine

Re:Apache Heron: A realtime, distributed, fault-tolerant str... by igetspam on Tuesday July 13, 2021 @09:19AM (#27820576)

We've been using Kafka for a while. We're thinking about ditching it for something AMQP. For all it's bells and whistles, it's just not really all that exciting. We're probably spending 5x more on Kafka than we could something much simpler.

[ Parent | on HN ]

Re:Apache Heron: A realtime, distributed, fault-tolerant str... by EdwardDiego on Tuesday July 13, 2021 @10:03AM (#27821104)

Kafka is only exciting when you're wanting to stream a metric shit ton of data without things falling over.

If you're using it, you're constrained by its engineering decisions, so you need to be sure it's a worthwhile tradeoff.

[ Parent | on HN ]

Re:Apache Heron: A realtime, distributed, fault-tolerant str... by jdmichal on Tuesday July 13, 2021 @09:46AM (#27820889)

We're doing the same. Actively moving away from Kafka, which was only ever used as a message queue anyway, to an actual message queue. We might move back at some point in the future to rebuild an actual event-driven processing model. But for now, it's not worth the additional complexity for the scale and systems we actually have.

[ Parent | on HN ]

Re:Apache Heron: A realtime, distributed, fault-tolerant str... by himoacs on Tuesday July 13, 2021 @11:00AM (#27821822)

Which product are you moving to?

[ Parent | on HN ]

Re:Apache Heron: A realtime, distributed, fault-tolerant str... by himoacs on Tuesday July 13, 2021 @10:57AM (#27821772)

Just to clarify...AMQP is a protocol so you will need a broker that supports it. The two options I see for you are RabbitMQ and Solace. Both support AMQP.

[ Parent | on HN ]

Re:Apache Heron: A realtime, distributed, fault-tolerant str... by marcelm72 on Tuesday July 13, 2021 @12:56PM (#27823302)

Apache Pulsar seems to have decent AMQP support also.

[ Parent | on HN ]