Re:NSQ – A realtime distributed messaging platform designed ...
by be_erik on Wednesday May 31, 2017 @11:33PM
(#14457797)
NSQ does nothing like this out of the box. You could create a channel for each topic that accumulates messages, but those messages would not be distributed across the cluster and after consuming them once they would be effectively gone. If that feature is a hard requirement NSQ wouldn't fit your use case.
Re:NSQ – A realtime distributed messaging platform designed ...
by StavrosK on Thursday June 01, 2017 @12:50PM
(#14461975)
I see, thank you. I'm a bit confused by "after consuming them once they would be effectively gone". Surely NSQ supports more than one subscribers, and won't remove messages after they've been delivered to a single client?
Re:NSQ – A realtime distributed messaging platform designed ...
by kasey_junk on Thursday June 01, 2017 @01:53PM
(#14462656)
It supports more than one subscriber as long as you have a separate channel for each. But if the channel gets created after some messages have been delivered, they will not be delivered to the new channel, only the messages after that.
To do replay with nsq (one way is) you'd attach their file consumer to its own channel as the first operation on a topic. Then in application code you'd need to read from that file whenever you need to do a replay (and you'd need to know when/where to read from/to).
Re:NSQ – A realtime distributed messaging platform designed ...
by StavrosK on Thursday June 01, 2017 @06:24PM
(#14465455)
That makes sense, thank you.
Re:NSQ – A realtime distributed messaging platform designed ...
by jehiah on Wednesday May 31, 2017 @08:45PM
(#14457083)
With NSQ there is a built in utility nsq_to_file which just becomes one additional consumer you'd use to archive each message topic to disk. It provides dead simple archiving of messages, but doesn't provide any native replay ability.
Another nice thing I've heard with Kafka is that it can store all messages since the beginning of time (if you want) and you can replay them to retrieve all your MQ-related state. Does NSQ do that, do you know?