News for hackers. Stuff that matters.
Re:System design hack: Postgres is a great pub/sub and job s... by ubu7737 on Friday November 08, 2019 @06:48PM (#21488198)
It makes the job-fetching more atomic, as the article points out properly. If you take >1, you have to sort the situation where 1 or more jobs fail, which adds an unreasonable amount of complexity.

On my first go I was taking batches, on the reasoning that it optimized for query performance. It was a complete disaster and I quickly realized that I was engaging in premature optimization.

Batching is out, stream-processing is in. If you design your job table correctly, PG will perform very well as an advanced stream processor.

There are no comments on this comment yet.