Re:Starting with Kafka
by lz400 on Saturday June 26, 2021 @09:02PM
(#27647549)
Using confluent platform community (free) gives you a working env with Kafka/Zookeeper/SchemaRegistry/Connect immediately without having to bother with setting up clusters, etc. Beyond that, yes, it's a pain to properly setup and I think most of the time you're looking at managed options unless you want to invest in a SRE team.
Re:Starting with Kafka
by tpetry on Sunday June 27, 2021 @02:57AM
(#27649255)
Isn‘t redpanda a reimplementation of kafka without the jvm and zookeeper? I mean how should they api compatible if they don‘t implement the same concepts.
Re:Starting with Kafka
by thibauts on Sunday June 27, 2021 @04:33AM
(#27649719)
This [1] could be a solution to most of author’s problem. Simple to use / few moving parts, very high perf, native HTTP produce and consume, clear and simple processing model with strong guarantees, among other things.
Re:Starting with Kafka
by winrid on Saturday June 26, 2021 @08:44PM
(#27647435)
Kafka Streams are also amazing, and would have solved many problems I've encountered in the past better than our microservices moving data around themselves.
Re:Starting with Kafka
by murukesh_s on Saturday June 26, 2021 @10:53PM
(#27648100)
Would you mind expanding the use cases you would have used Kafka streams for? And why no other solutions could have helped? (Performance or complexity etc)
Where did that idea come from? He literally wrote how they did it without streams by moving data between their Microservices, likely with synchronous http calls. He would've just preferred to just write a stream instead of the services.
Re:Starting with Kafka
by carlosap on Monday June 28, 2021 @11:09AM
(#27662942)
There have been like three iterations until we've been able to actually release an stable integration that deals well with high traffic. All together, almost 6 months.
Re:Starting with Kafka
by ElViajero on Sunday June 27, 2021 @03:09AM
(#27649316)
I guess that it is part of the curse of knowledge.
Even the article starts by
"I just want to share my thoughts on Kafka after using it for a few months, always from a practical point of view. I don’t know anything more than the basics about Kafka internals. "
It is just confusing for anyone that does not know that Apache Kafka is a software application to process queues.
The article is named "Starting with Kafka" but assumes that you have knowledge of what it is, its use cases, and other not-so-basic information.
Re:Starting with Kafka
by ofrzeta on Sunday June 27, 2021 @03:31AM
(#27649433)
I think if someone were to write an article on the writer Kafka they would not phrase the headline "Starting with ... " but something like "Reading Kafka" or "How to read Kafka". I am not a native speaker, though.
As I understand it, this is where Redpanda enters the picture. Maybe others here disagree?
https://softwareengineeringdaily.com/2021/01/22/redpanda-kaf...
https://github.com/vectorizedio/redpanda
https://vectorized.io/
[1] https://github.com/dataptive/styx
Where did that idea come from? He literally wrote how they did it without streams by moving data between their Microservices, likely with synchronous http calls. He would've just preferred to just write a stream instead of the services.
Even the article starts by
"I just want to share my thoughts on Kafka after using it for a few months, always from a practical point of view. I don’t know anything more than the basics about Kafka internals. "
It is just confusing for anyone that does not know that Apache Kafka is a software application to process queues.
The article is named "Starting with Kafka" but assumes that you have knowledge of what it is, its use cases, and other not-so-basic information.