Do you have more details how that's done?
I assume there are some limitations if you actually skip the LB? How do the host NICs rewrite the LB address, does this imply there is hardware support for this kind of bypass routing?
Gcp for example has the potential for ~1 k revenue per core over a system lifespan. A smart nic is probably ~1.5k, so saving 2 cores outs you in the black and has other security advantages.
In typical data centres the "network" is really just a handful of Cisco boxes. In the cloud, the network extends to the FPGAs or ASICs in the servers themselves, including the hypervisors.
When a packet leaves a VM, the hypervisor host rewrites it, typically in hardware, and then when the remote hypervisor receives it, the packet is rewritten back to what the destination VM accepts.
This allows thousands of overlapping 10.0.0.0/24 subnets, and "tricks" like direct VM-to-VM traffic that appears to go via a load balancer.
The actual load balancer VMs just "set up" the flow, while instructing the hosts to take over the direct traffic in their stead.
I guess they're seen high amounts of out-of-order packets and there's some detailed write ups on why that happens with GCP SDN implementation.
For example in Azure, only the first two packets in a VM->LB->VM flow will traverse the LB. Subsequent packets are direct from VM-to-VM and are rewritten in the host NICs to merely appear to go via the LB address. This enables staggering throughputs that no “software in a VM” can possibly hope to match.
Personally I wish people would stop with the unnecessary middle boxes. It’s 2023 and there are cloud VMs now that can put out 200 Gbps! Anything in path of a few dozen such VMs will melt into slag.
This is especially important for Kubernetes and microservices in general that are already very chatty and have layers of reverse proxies five deep in surprisingly common configurations already.