Do you really need Redis? How to get away with just PostgreSQL

Re:Do you really need Redis? How to get away with just Postg... by jffry on Saturday June 12, 2021 @10:58AM (#27484919)

Not sure. Maybe something else is responsible for identifying "stuck" jobs and kicking them out to humans to decide what to do (assuming worker crashes are rare)

If we're talking PostgreSQL specifically, and newer-ish Postgres (9.5+ I think), then you can leverage its abilities to do all this in one atomic query:

  UPDATE jobs
  SET status='working'
  WHERE id = (SELECT id
              FROM jobs
              WHERE status=NULL
              LIMIT 1
              FOR UPDATE
              SKIP LOCKED)
  RETURNING id

[ Parent | on HN ]

Re:Do you really need Redis? How to get away with just Postg... by shawnz on Saturday June 12, 2021 @11:00AM (#27484931)

If you are willing to use SELECT ... FOR UPDATE SKIP LOCKED, then you can implement the queue without any UPDATE query or status column at all. Just lock the record to mark it as in progress, that is how you get the benefit that when the worker crashes, it will automatically be returned to the queue.

[ Parent | on HN ]