If I synchronously hit some service then I need to build facilities into the service to say that it's too heavily loaded, and perhaps handle those responses on the client.
If I pop some job onto a message queue or asynchronous endpoint, things will just slow down for a while. Which is normally as good a way as any as soaking up load.
> If I synchronously hit some service then I need to build facilities into the service to say that it's too heavily loaded, and perhaps handle those responses on the client.
Most of the time, you have to have timeouts when there is a synchronous API across the network.
In a large system ideally you'd want to reflect the level of loading back to the input source so the source can slow down sending the data. Think of TCP, tcp works this way on a small scale. One way to fix the problem is to leverage that to actually open a TCP stream and send data that way. The sender will slow down accordingly.
Now if you know that your load is not constant, so there are periods of high activity when inputs are generated then you can try and absorb (and amortize) those high volume peaks using asynchronous queues.
This is perhaps a point one cannot get to theoretically just sort of thinking about. This realization comes after observing the effects in practice. It is interesting to observe distributed systems, especially the ones that rely on asynchronous messages, thing go wrong and queues start filling up. Or even weirder machines start to synchronize for some reason. Like the size of the queues will oscillate in resonance of some sorts.
One way out is to reduce asynchronicity but that comes with serious performance penalties.