News for hackers. Stuff that matters.
Re:Why is load balancing gRPC tricky? by thayne on Friday May 14, 2021 @03:08AM (#27151608)
> Although this feature from ALBs is relatively new but the arguments for sticky gRPC connections and NLB are also applicable with ALBs. If yu end up with a single sticky and persistent connection from a client that starts sending a large volume of requests, you are overloading a single server instance that holds that connection and the requests are not being load balanced as they should.

That seems strange to me. Without end-to-end http/2, ALBs would load balance individual requests, not the entire TCP session, even with keepalives. And it sounds like individual messages on the same connection can be routed to different target groups, which would make it even more stange not to balance individual requests.

Re:Why is load balancing gRPC tricky? by 1787 on Friday May 14, 2021 @08:40AM (#27153587)
Yeah, I think the post author may be confused. I've migrated several high traffic gRPC services (basically TensorFlow Serving) from AWS NLBs to ALBs for the improved load balancing. It worked pretty much as desired (although a little expensive).
Re:Why is load balancing gRPC tricky? by jayd16 on Friday May 14, 2021 @09:25AM (#27154023)
I don't think grpc even supports http/1.1 on all (any?) languages so it was never a feasible solution for grpc.
Re:Why is load balancing gRPC tricky? by thayne on Friday May 14, 2021 @11:17AM (#27155179)
Right. I was talking about normal http/1.1 traffic with http keepalive.