News for hackers. Stuff that matters.

AutoMQ: A Cloud-Native Kafka Alternative with 95% Cost Savings over Confluent

Apache Kafka is critical for real-time streaming, but its costs, especially with Confluent, can be prohibitive.

AutoMQ offers a fresh take: a cloud-native Kafka implementation that reduces costs by rethinking traditional architectures.

Preconditions for Comparison

Peak Throughput: 0.1 GB/s

Average Throughput: 0.01 GB/s

Monthly Data Transfer: 25,920 GB

Storage Volume: 2,592 GB

Architecture: Multi-AZ (AWS)

Data Retention: 3 Days

Key Cost Differences:

Confluent Total: $17,369/month

AutoMQ Total: $718/month (95.8% less) How?

Stateless Brokers: No shared-nothing overhead.

Shared Storage: Amazon S3 eliminates cross-AZ replication costs.

Elastic Scaling: Decouples compute and storage for efficiency.

Core Benefits: Massive cost savings: Compute (-94.7%), Network (-99%), Storage (-78.3%).

Simpler operations with cloud-native scaling.

Retains Kafka's core performance without heavy infrastructure.

For anyone working with Kafka: Is this a sustainable solution for cost reduction?

Would you consider migrating to a cloud-native Kafka setup like AutoMQ?

( 4 comments | 2 points | on HN )
Re:AutoMQ: A Cloud-Native Kafka Alternative with 95% Cost Sa... by jareds on Wednesday December 18, 2024 @07:02PM (#42456852)
As someone who works with Confluent Kafka I'm not happy with there pricing structure. While the cost savings look good I'd need a much larger Confluent bill to consider moving off Confluent. The cost savings appears to be about the cost of a developer. With complex systems using many micro services to communicate with Kafka the switching cost is not worth it at a small scale given the business values developers could provide instead of doing the switch. When your talking about much larger data volumes or multiple clusters across multiple regions then the math starts to look better.
Re:AutoMQ: A Cloud-Native Kafka Alternative with 95% Cost Sa... by jackbauer24 on Wednesday December 18, 2024 @10:51PM (#42458268)
I agree with you. If we have more data, it can definitely be more cost-effective.
Re:AutoMQ: A Cloud-Native Kafka Alternative with 95% Cost Sa... by BitNomad on Wednesday December 18, 2024 @10:48PM (#42458251)
Good insights. I also find the complete blog: https://medium.com/@brij.python/automq-the-affordable-kafka-...
Re:AutoMQ: A Cloud-Native Kafka Alternative with 95% Cost Sa... by dont_click_here on Friday December 20, 2024 @05:21PM (#42475665)
This is a sponsored marketing campaign targeting Confluent with coordinated actions across multiple platforms. It's a bunch of AI-generated noise from a LinkedIn "top voice".

Brij Kishore Pandey (aka Honestsoul), a Principal Engineer at ADP, authored an sponsored Medium article promoting AutoMQ that claims 95.8% cost savings over Confluent. Brij shard the content here under the username "Honestsoul," a recently created account with no prior activity (but also his GitHub username).

The coordinated nature of this campaign becomes apparent through several elements. 1. AutoMQ's CEO, Xiaorui Wang, who sponsored the content, participated in the Hacker News discussion without disclosing his relationship to the company (jackbauer24). jackbauer24 identifies himself as such in his comments. 2. The post's distribution pattern shows strategic amplification, with a commenter named "BitNomad" directing readers to the full Medium article without solicitation. 3. Brij Pandey's GitHub profile displays patterns suggesting coordinated follower activity. He has only 47 followers on GitHub and several of those accounts were only set up to follow him and his projects. I question how many of his 547,082 LinkedIn followers are real.

Brij's article presents a specific workload scenario comparing AutoMQ's $718/month cost against Confluent's $17,369/month, focusing on architectural differences in storage and compute resources. The technical claims focus on AutoMQ's cloud-native architecture using Amazon S3 for storage and stateless brokers, contrasting with Confluent's traditional shared-nothing architecture. While the cost comparison presents dramatic savings, it's based on a specific workload scenario with particular throughput, storage, and retention parameters that may not represent typical enterprise usage patterns.

And the cherry on top of it all is that the content for this article gets flagged as AI generated content in both Ghostbuster.app and hgptzero.me. I suggest that Xiaorui ask for his money back.

References: https://www.linkedin.com/in/brijpandeyji/ https://github.com/honestsoul https://news.ycombinator.com/threads?id=jackbauer24 https://www.linkedin.com/in/vintagewang/