https://ferd.ca/handling-overload.html
I don’t find that kind of fully async processing pipeline very idiomatic TBH, at least in Erlang - more typically, every request would be its own actor, and do blocking requests to other actors that own shared resources.
The idea of each request being its own actor being more idiomatic is intriguing. Naively though does it really solve the problem? Don't you just end up with an overflow of actors now instead of an overflow of one actor's inbox?
That being said the main reason an Erlang program will use an actor per request is not to prevent overload, it’s for fault tolerance; in your a -> b -> c -> d example a bug (e.g. uncaught exception) triggered by one request that happens in process c can cause all requests in flight to fail, while with a process per request there’s only one affected.
It does mean you may have thousands of actors rather than four or so, but that’s not a problem since Erlang is built and optimized for that kind of workload. On a platform where actors are OS threads, it may make sense to use a different approach.
However, I also don't think you have to worry about the message payloads being too large. In Erlang, normally objects are copied between processes. However, for large objects (>64 bytes) they are put in a shared heap, and only the pointers are copied between processes[0].
[0] https://hamidreza-s.github.io/erlang%20garbage%20collection%...
Check out this video which describes it pretty well: https://youtu.be/srtMWzyqdp8?t=1027
I don't see the manifestation of this problem as forfeiting the point. You could on the opposite extreme write one monolithic application that does everything: web interface, application logic, and data storage. It's readily apparent that for most use-cases, the benefits of separating, say, your web and database software will more than outweigh the added complexity cost of running two applications and routing requests between them. (And we don't anymore balk at running pools of web servers in front of our databases.) Continue to chop your application into smaller segments and of course you'll see diminishing returns. The application just has to be complex enough to raise the expectation of benefit from segmentation.
If you consider a simple chain of actors that do some work and then send on a result to a next actor that then does some more work:
Suppose something (anything) happens that make messages to B take slightly longer to process than the rate at which A is sending them. The result is that B's message inbox expands indefinitely. One of two things will happen depending on the queuing model: either A will block on send because B refuses to accept another message, or B will simply run out of memory from too many pending messages.This of course, is just the linear scenario. Consider a complex network where there may even be cycles (something D does could somehow indirectly result in a message coming back to A). You've essentially got a completely unstable, unpredictable system. The only way to avoid it is to ensure that every actor is massively over-resourced so that it can never fall behind on processing its messages.
Of course, the second stage of this is that you start designing scalable actor "pools" where more than one actor can service messages and you can scale the size of the pool up and down. But this is at the cost of adding even more dynamic, unpredictable behaviour, and to some extent you just lost the simplicity you were trying to gain by using Actors in the first place.
I'm curious if any thoughts from more experienced folks about how to handle these issues!