Re:Random Load Balancing Is Unevenly Distributed
by joosters on Thursday August 31, 2023 @12:09PM
(#37339465)
It's surprising that "least requests" is still so less commonly used than random or round-robin. It's a simple metric and yet it handles work-sharing remarkably well, without any need for servers to communicate their load back to the LB. (As you correctly say, the LB needs to be aware of idle/keep-alive connections in order to balance properly)
[Also, to be pedantic, it should really be called "fewest requests" - but I've never seen a product call it that!]
AWS ALB (and others I'm sure) can balance by "Least outstanding requests" [1] which means a server with the least in-flight HTTP requests (not keep-alive network connections!) to an app server will be chosen.
If the balancer operates on the network level (eg NLB) and it maintains keep-alive connections to servers, the balancing won't be as even from a HTTP request perspective because a keep-alive connection may or may not be processing a request right now and so the request will be routed based on number of TCP connections to app servers, not current HTTP request activity to them.
[1] https://docs.aws.amazon.com/elasticloadbalancing/latest/appl...