Google pub/sub released an ordering feature

Re:Google pub/sub released an ordering feature by chimera77 on Friday October 02, 2020 @02:15AM (#24659486)

There are a couple of options without needing guaranteed ordering:

- jobs can have ever increasing ids, workers record the last seen id in one place, and ignore jobs with ids less than last seen

- job results are returned for each job to a supervisor. if a job result doesn't match current expected state, resend job. jobs should be idempotent in case a job is sent multiple times

If the create job is expensive, the latter solution would be less ideal, though.

[ Parent | on HN ]

Re:Google pub/sub released an ordering feature by manigandham on Friday October 02, 2020 @03:27AM (#24659975)

Just ignoring tasks isn't helpful, it means you still process things out of order and just never get to the previous pending item in that case.

Much easier to just have strict order and process it off the line as it comes.

[ Parent | on HN ]

Re:Google pub/sub released an ordering feature by bgdam on Friday October 02, 2020 @03:31AM (#24660013)

Creating ever increasing ids reliably at scale is not trivial. You will probably end up having a single server generating these ids which will then become a single point of failure.

[ Parent | on HN ]

Re:Google pub/sub released an ordering feature by caturopath on Friday October 02, 2020 @11:53AM (#24663879)

(Or a distributed system - this is no trickier to migrate away from SPOF than anything else with global state.)

[ Parent | on HN ]