Re:How to Build Software Like an SRE
by benlivengood on Monday October 17, 2022 @06:41PM
(#33240438)
There's never a reason that a human needs to read an error message caused merely by Kubernetes restarting a pod in the normal course of operations.
Passing every error up the stack causes people to be confused by 500 status errors for trivially retry-able calls. Browsers (and TCP) natively implement retries despite there being no SLAs in place.
If the problem is just that the 99 calls could overload downstream services on retries, the ideal solution is to add rate-limiting, though admittedly that is an imprecise science.