Anything can be a message queue if you use it wrongly enough

Re:Anything can be a message queue if you use it wrongly enough by Ozzie_osman on Sunday June 04, 2023 @01:17PM (#36187408)

This is actually pretty common, and usually a "good enough" solution. You can also add things like scheduling (add a run_at column), at least once execution (mark a row when it is being processed, delete it only when successful), topics, etc with minor modifications to your table.

If you want something that works "well enough" I'd say it's a reasonable choice.

[ Parent | on HN ]

Re:Anything can be a message queue if you use it wrongly enough by larperdoodle on Sunday June 04, 2023 @01:36PM (#36187580)

Yeah, I'm using it as a transactional outbox to ensure at least once delivery to SNS.

Can't really think of a better way to ensure that a message is always sent if the DB transactions succeeds and is never sent if the DB transaction fails

[ Parent | on HN ]

Re:Anything can be a message queue if you use it wrongly enough by Ozzie_osman on Sunday June 04, 2023 @01:52PM (#36187728)

You can get so far by ensuring at least once and making everything idempotent (will get you as close to "exactly once" as you can). With a database, the most common pattern is: insert the row for the job, when a worker starts working on it, mark it as in progress so it doesn't get started again, if the task fails, or after some reasonable time-out period, another worker can pick up the task again, ultimately the row for the task is only ever deleted when a worker successfully completes it.

[ Parent | on HN ]