News for hackers. Stuff that matters.
Re:Anything can be a message queue if you use it wrongly enough by wolfgang42 on Sunday June 04, 2023 @04:07PM (#36189090)
Database engines are getting features like SELECT FOR UPDATE SKIP LOCKED, so what were once serious blockers on this idea may no longer be as much of a problem.
Re:Anything can be a message queue if you use it wrongly enough by scarface74 on Sunday June 04, 2023 @04:15PM (#36189161)
And if you skip records that you are depending on for your poor man’s queue, aren’t you just hiding bugs?
Re:Anything can be a message queue if you use it wrongly enough by Spivak on Sunday June 04, 2023 @04:39PM (#36189365)
No? Unless there's some edge case with that statement I don't know about. That statement is basically tailor made for queues so you can select jobs that aren't currently being worked on by other workers.

Inasmuch as you trust your db's locking correctness it eliminates the concurrency issues. You can very naively have n workers pulling jobs from a queue not stepping on each-other.

Re:Anything can be a message queue if you use it wrongly enough by wombatpm on Sunday June 04, 2023 @10:17PM (#36191524)
Is that necessary if you can just have your processes handle locking themselves?
Re:Anything can be a message queue if you use it wrongly enough by wolfgang42 on Monday June 05, 2023 @12:05AM (#36192118)
It’s not necessary, but it is a lot less fiddly: you automatically look at only the tasks that someone else isn’t currently working on, and because the lock is held by the database connection you get automatic retries if your worker crashes and drops the connection. You could figure out all of the interactions needed to make this work yourself, but if the database already has support built in you may as well use it (and there’s a straightforward path to migrate if you need more sophistication later).