News for hackers. Stuff that matters.
Re:How far can you go without a message queue? by worldsayshi on Thursday August 25, 2022 @08:40AM (#32592921)
"But you can go very far without them by using the DB or redis as your message queue."

So, you can go very far without a message queue... by using a simpler message queue?

Re:How far can you go without a message queue? by mhuffman on Thursday August 25, 2022 @09:20AM (#32593336)
I mean, at some point you have to consider something a "proper" message queue. Otherwise, at the lower extreme your web server would act as a message queue.
Re:How far can you go without a message queue? by shapefrog on Thursday August 25, 2022 @09:50AM (#32593694)
"Redis is an open source (BSD licensed), in-memory data structure store, used as a database, cache, and message broker"

Lets draw the line somewhere the other side of applications that advertise themselves as having that function?

Re:How far can you go without a message queue? by Spivak on Thursday August 25, 2022 @06:18PM (#32600656)
The number of people that think that Redis is a KV store and nothing else is too damn high.
Re:How far can you go without a message queue? by skyde on Friday August 26, 2022 @02:11AM (#32603875)
Redis is not a database. it doesn’t have proper transaction support.

it’s basically a cache with a fancy API so you can incrementally update data structure instead of having to constantly write the whole thing you are trying to cache.

Re:How far can you go without a message queue? by Spivak on Friday August 26, 2022 @11:20AM (#32608676)
Sure it is! Redis is even fully ACID if you use the right persistence settings complete with transactions support and a WAL. You can even set up locking to abort/rollback transactions. In DB parlance Redis’ isolation level is strict serializability which is stronger than Postgres.

If you mean “it doesn’t have every feature of Postgres” and behaves a little differently then sure but it absolutely is a database.

Re:How far can you go without a message queue? by skyde on Monday August 29, 2022 @01:31PM (#32640980)
the only way to do an "atomic increment operation" is to use the "Watch" command or to write stored procedure.

Watch is a poor man transaction and stored procedure does not work when you need to read value interactively from somewhere else (user/filesystem/another database) while inside a transaction.

Re:How far can you go without a message queue? by Spivak on Tuesday August 30, 2022 @06:44PM (#32656120)
You're literally describing INCR, it's atomic. But I know what you mean, you want atomic GET then SET which is done by WATCH.

I don't disagree with you about WATCH being less powerful than PG transactions. SQLite data types are less powerful than PG types but it's still a database.

Re:How far can you go without a message queue? by dropofwill on Thursday August 25, 2022 @10:00AM (#32593819)
Actually a more complicated message queue, because it's on you to implement all the queueing logic. Then back it by a potentially non-durable data store (its very tricky to get redis durable and ha) with no monitoring?

All of this to avoid spinning up an SQS queue?

Re:How far can you go without a message queue? by worldsayshi on Thursday August 25, 2022 @10:19AM (#32594040)
Isn't that a bit of an odd wording though. Of course if you only buy the horse without the cart you have to build the cart, making "your" solution more complicated. But you still bought the "simpler" solution i.e. the horse without a cart.
Re:How far can you go without a message queue? by dropofwill on Thursday August 25, 2022 @11:46AM (#32595346)
Complexity may be the wrong word, but what I mean is surface area for bugs you are responsible for.

I also question whether you really bought a horse in this analogy.

Re:How far can you go without a message queue? by treis on Thursday August 25, 2022 @10:50AM (#32594408)
By using something you're already using.