News for hackers. Stuff that matters.
Re:RabbitMQ Streams Overview by growup12345 on Tuesday July 13, 2021 @05:25PM (#27826509)
It’s not a queue but an append only log
Re:RabbitMQ Streams Overview by saltedonion on Tuesday July 13, 2021 @06:05PM (#27826850)
Redis too have an append only log at-least for backup and persistence. What’s the difference with rabbitmq?
Re:RabbitMQ Streams Overview by spenczar5 on Tuesday July 13, 2021 @10:53PM (#27829083)
RabbitMQ already had a queue - that's it's main feature. This adds something which is not a queue; it's an append-only log.

Queues let many writers put messages into a single topic. Then, many readers come to that topic, and pop messages off. Each message goes to just one reader. You could use this for background jobs. For example, if you were running YouTube, you might handle uploads this way: the uploaded videos are put into a queue, and workers process them to transcode them and make them playable on the website. You don't want the uploads to be processed more than once.

Append-only logs let many writers append messages onto a single topic. Then, many readers may replay the entire history of messages whenever they want. Each message may go to many readers, and may even go to the same reader multiple times if they want. You could use this to build a "message bus" where you want lots of things to happen after an action. For example, every time a user "likes" something on facebook, maybe we want to notify the content-producer, notify their friends, and update some recommendation algorithms - three different things that we want to do each time, and we don't want errors in one to block the others.

Re:RabbitMQ Streams Overview by TwoHeadedBeast on Wednesday July 14, 2021 @07:12AM (#27831795)
There is already a mechanism suitable for your Facebook example use-case, namely publishing to an exchange which will send the message to multiple queues.
Re:RabbitMQ Streams Overview by spenczar5 on Wednesday July 14, 2021 @09:01AM (#27832533)
Right. The big feature that append only logs allow is replay. I have never really seen the point, though - and I say this as a big Kafka user!
Re:RabbitMQ Streams Overview by hosh on Tuesday July 13, 2021 @07:43PM (#27827582)
It looks like Rabbitmq stream semantics matches up better with Kafka. Redis’s stream is more lighter weight.
Re:RabbitMQ Streams Overview by exdsq on Tuesday July 13, 2021 @05:59PM (#27826803)
I thought append only logs required crypto tokens to use nowadays?
Re:RabbitMQ Streams Overview by spicybright on Tuesday July 13, 2021 @08:12PM (#27827808)
I'm positive it's used in crypto, but there's nothing novel about append only logs. It just means you don't remove the old stuff.

To elaborate a bit, crypto is backed by tokens specifically to ensure integrity in a distributed network (on machines you don't own)

If you own the DB you don't need this.

Re:RabbitMQ Streams Overview by nivertech on Wednesday July 14, 2021 @02:15AM (#27830209)
Of course you can pay for each new log entry using crypto tokens, but you don't have to.

You can use an open source version of AOL (Append-Only Log) SW for free, alternatively you can use a managed cloud version, but there you will have to pay with something called USD which is a social construct 1:1 pegged to a crypto-tokens like Tether USDT, MakerDAO DAI, and other so called "stablecoins" ;)

Re:RabbitMQ Streams Overview by exdsq on Wednesday July 14, 2021 @03:37AM (#27830627)
I guess I have to add /s explicitly nowadays!