News for hackers. Stuff that matters.
Re:Apache Kafka Goes 1.0 by manigandham on Wednesday November 01, 2017 @03:50PM (#15604853)
I recommend reading the Pulsar architecture page: https://pulsar.apache.org/docs/latest/getting-started/Concep...

Biggest difference is that Pulsar splits compute from storage. Pulsar brokers are stateless services that manage topics and handle clients (publishers and consumers) while reading/writing data from a lower tier of bookies, which is the Apache BookKeeper project for low-latency real-time storage workloads.

Better performance and easier scaling compared to Kafka's partition rebalancing. It also natively supports data center replication and multiple clusters working together globally. Pulsar addressing is property/cluster/namespace/topic so multi-tenant isolation is built-in.

Anywhere you use Kafka can use Pulsar and also consolidate pub/sub only systems that need lower latency than Kafka.

Re:Apache Kafka Goes 1.0 by lmsp on Wednesday November 01, 2017 @09:07PM (#15607012)
I think it is beyond partition rebalancing. There is a fact that people didn't realize of making message broker `stateless`. It is actually much better on reacting to failures or shifting traffic, which is critical when running a messaging bus for online services. because it doesn't have to wait for copying the data of a whole partition when error occurs.
Re:Apache Kafka Goes 1.0 by manigandham on Wednesday November 01, 2017 @11:04PM (#15607515)
Agreed, the architecture page and other blog posts do a more thorough job of explaining the details. Having a stateless broker layer on top of a focused data layer makes all the operations much easier. I expect BookKeeper to further integrate with the various cloud storage APIs so it can also start to become stateless cache.