Ask HN: How do you handle duplicate side effects when jobs, workflows retry?

Re:Ask HN: How do you handle duplicate side effects when job... by moomoo11 on Thursday February 26, 2026 @10:45PM (#47176225)

You proxy those api calls yourself and have idempotency to cover you for those APIs that don’t have it. If you architect it right you won’t have more than a ms latency addition. You can avoid the race condition issues by using atomic records so if something else tries they’d see it’s in progress and exit.

[ Parent | on HN ]

Re:Ask HN: How do you handle duplicate side effects when job... by shineDaPoker on Friday February 27, 2026 @04:15AM (#47178415)

This is exactly the approach I took. Proxy layer that: - Uses atomic records (fence tokens) to prevent concurrent execution - Checks external system first before retrying (the retrieval step) - Records result for future lookups

The atomic records part is critical - I learned the hard way that just checking a DB flag isn't enough (process can freeze between check and execute, lease expires, another process takes over, both execute).

How do you handle the case where: 1. Process acquires atomic lock 2. Calls external API successfully 3. Process freezes before releasing lock 4. Lock expires, new process acquires it 5. New process calls API again → duplicate

Do you just accept this edge case (rare but possible)? Or is there a mitigation I'm missing?

[ Parent | on HN ]

Re:Ask HN: How do you handle duplicate side effects when job... by moomoo11 on Friday February 27, 2026 @05:50PM (#47186910)

yeah, this is a common edge case in this scenario and as you say it will be rare but possible.

however i don't think its that complicated (idk your system obv, so I could be off base) but i think your best bet is to not retry blindly and instead just query to see if already happened. If it already happened you're good and if not you handle it appropriately.

considering it will be rare i don't think handling it how i suggest it will affect you that badly. keep it simple should be your priority imo with these kinds of operations.

[ Parent | on HN ]