Ask HN: How do you prevent retry cascades in LLM systems?
We ran into a retry amplification issue in one of our LLM agents recently.The provider returned 429s for a short period. We had per-call retry limits in place. We did NOT have containment at the request-chain level.Because calls were nested and spread across multiple workers, retries multiplied in ways we didn’t anticipate.Per-call limits were not enough.For those running LLM systems in...