News for hackers. Stuff that matters.
Re:Using load shedding to survive a success disaster - CRE l... by drdrey on Monday December 19, 2016 @06:57PM (#13216130)
Some additional things that can be done:

* soft-shedding, where instead of dropping a request (which might just incur a retry storm), sometimes it is appropriate to send back a cheap response so that the clients sees a successful response instead of an error

* route critical requests and non-critical requests to separate clusters that can be scaled and configured independently. The blog post mention doing that using DNS, but that also works for mid-tier services.

* build back-pressure into the client. Instead of a timeout or error, a well-conforming client can enter "polite" mode when it receives a signal that the backend is overwhelmed.

Re:Using load shedding to survive a success disaster - CRE l... by euyyn on Monday December 19, 2016 @07:02PM (#13216157)
Is soft-shedding a good idea? You're still paying CPU time to come up with a response that is a 200, but probably won't fool the client or the user anyway. Without a signal that the server is having problems with load, the retries can be more aggressive.
Re:Using load shedding to survive a success disaster - CRE l... by dantiberian on Monday December 19, 2016 @08:43PM (#13216639)
Imagine a Netflix type service which needed to calculate a bunch of different categories to show the user. If some of them are more expensive to compute than others, those categories could be dropped, while still returning an acceptable response to the user.
Re:Using load shedding to survive a success disaster - CRE l... by drdrey on Tuesday December 20, 2016 @01:19PM (#13222415)
It makes sense if CPU time is not your most constrained resource. For instance, you might use it to avoid doing expensive remote calls.
Re:Using load shedding to survive a success disaster - CRE l... by toomuchtodo on Monday December 19, 2016 @07:34PM (#13216333)
That soft response should be a 429, with the client performing exponential backoff.