News for hackers. Stuff that matters.

A comment on Do you really need Redis? How to get away with just PostgreSQL. Parent comment

Re:Do you really need Redis? How to get away with just Postg... by layoutIfNeeded on Saturday June 12, 2021 @06:49AM (#27483500)
Doesn’t sound like a robust solution. What if the job takes longer than your estimated expire time? Does the worker keep updating the expiry date on a background thread while doing the processing?
[ Parent | on HN ]
Re:Do you really need Redis? How to get away with just Postg... by cbo100 on Saturday June 12, 2021 @08:29AM (#27483991)
Same for any queuing system. You need to set the expiry time long enough for the expected task duration.

In SQS, for example, you use a visibility timeout set high enough that you will have time to finish the job and delete the message before SQS hand it off to another reader.

You won’t always finish in time though, so ideally jobs are idempotent.

[ Parent | on HN ]