Pulsar vs. Kafka

Re:Pulsar vs. Kafka by SmooL on Friday July 10, 2020 @09:34AM (#23791305)

Is there any sort of 'single node' version of these frameworks? I'm very interested in building event-driven solutions, but I don't need the scale offered by kakfa/pulsar, and I really don't want all the complexity. Is there any reason nobody has made a smaller, less distributed event-centric DB?

[ Parent | on HN ]

Re:Pulsar vs. Kafka by leourbina on Friday July 10, 2020 @09:39AM (#23791353)

Redis Streams is a pretty good lightweight replacement for Kafka. Of course, what Kafka gets you are durability, being able to pick among at-least-once, at-most-once and exactly-once delivery of messages, among other things

[ Parent | on HN ]

Re:Pulsar vs. Kafka by SmooL on Friday July 10, 2020 @10:04AM (#23791550)

Thank you, I'll check it out

[ Parent | on HN ]

Re:Pulsar vs. Kafka by Tsarbomb on Friday July 10, 2020 @09:45AM (#23791400)

You can run kafka on a single node no problem.

Give this a try: https://www.digitalocean.com/community/tutorials/how-to-inst...

[ Parent | on HN ]

Re:Pulsar vs. Kafka by oweiler on Friday July 10, 2020 @01:21PM (#23793886)

Single node Kafka still needs Zookeeper, though.

[ Parent | on HN ]

Re:Pulsar vs. Kafka by SmooL on Friday July 10, 2020 @10:03AM (#23791544)

Oh cool, thank you!

[ Parent | on HN ]

Re:Pulsar vs. Kafka by itestyourcode on Friday July 10, 2020 @10:08AM (#23791586)

Pulsar does offer a standalone mode. It has a vertical stack of Pulsar broker, bookkeeper, and zookeeper in one process. The standalone mode also comes in a single docker image.

However, the complexity you refer to is essential in a reliable messaging framework. Use of zookeeper or any log consensus algorithm requires multiple nodes ( 3 or more) to achieve durability and high availability goal. It is out of necessity. This is actually essential complexity.

There is another messaging framework called Nats.io. It is not persistent so architecturally relatively simpler. You might want to investigate.

[ Parent | on HN ]

Re:Pulsar vs. Kafka by gerbal on Friday July 10, 2020 @09:48AM (#23791424)

Postgres can be used as an effective event centric database and pub/sub queue with NOTIFY and LISTEN

[ Parent | on HN ]

Re:Pulsar vs. Kafka by addisonj on Friday July 10, 2020 @05:14PM (#23796548)

Pulsar offers a standalone mode, which runs everything inside a single JVM and you can just backup the data files. Pulsar also can be made to run with multiple components, but not distributed.

[ Parent | on HN ]

Re:Pulsar vs. Kafka by lymitshn on Friday July 10, 2020 @03:34PM (#23795424)

nats streaming worth checking out, or if you dont care about the streaming part nats/zeromq should be simple enough (simple pub/sub infra tools).

[ Parent | on HN ]

Re:Pulsar vs. Kafka by staticassertion on Friday July 10, 2020 @09:38AM (#23791341)

Redis?

[ Parent | on HN ]