New: Streams are append only, but I push eta tasks onto the front of a queue when their time is ready to run. Looks like streams could work and might even be better suited instead of lists, but I would have to figure out some way to run eta tasks before other pending tasks in a queue.

Old when I confused streams with pubsub: Streams aren't durable... a worker goes offline for a second and all of a sudden you lost all your pending tasks. Redis lists are durable, so you can add tasks first then boot up a worker later and the worker will process those waiting tasks. With streams, the worker wouldn't see any pending tasks when it boots.

AFAIK, streams are treated as any Redis data type and stored on disk making them durable. Redis even allows you to use an append only file and the database snapshot for persistence/durability with differing levels of requirements (eg. every operation, every n seconds, never). Though I'm not sure if the streams are handled the same in terms of high availability though...

Also while plain XREAD doesn't give you pending tasks, using consumer groups and look at XREADGROUP (specifically using ID = 0 to get all pending tasks) should get you all pending tasks before continuing onto unseen tasks. There is also XCLAIM and XAUTOCLAIM which you can filter the pending tasks by how long they have gone unacknowledged for and have another worker claim the tasks.

Oh sorry, somehow I thought we were talking about pubsub.

It is a good question. Maybe because they aren't conceptually as simple as lists and require recent versions of redis. And maybe because xstream is persisting which means they use the disk and lists are not (unless you have incremental dumps turned on)

Why not just use a simple FTP server with curlftps to replace Dropbox then? ;)

But honestly, it seems like you did not read the article at all. That’s seems to be exactly what they do for the queue part, but you still need to manage the workers executing the tasks, lifecycle management and interfacing/serialization/deserialization for the tasks and results

From the guidelines:

> Please don't comment on whether someone read an article.