Re:Google shares software network load balancer design power...
by asuffield on Wednesday March 16, 2016 @07:33PM
(#11301517)
(Tedious disclaimer: my opinion only, not speaking for anybody else. I'm an SRE at Google, and I'm oncall for this service.)
On the understanding that I can't tell you any more than what it already says in the paper, the key detail that you're looking for is section 4.1.2:
"""
Maglev originally used the Linux kernel network stack
for packet processing. It had to interact with the NIC using
kernel sockets, which brought significant overhead to
packet processing including hardware and software interrupts,
context switches and system calls [26]. Each
packet also had to be copied from kernel to userspace and
back again, which incurred additional overhead. Maglev
does not require a TCP/IP stack, but only needs to find a
proper backend for each packet and encapsulate it using
GRE. Therefore we lost no functionality and greatly improved
performance when we introduced the kernel bypass
mechanism – the throughput of each Maglev machine
is improved by more than a factor of five.
"""
My understanding is that RPS just steers packets into the Linux TCP stack. Nothing on the maglev machine wants to process the TCP layer; that workload can be distributed over the backends.
On the understanding that I can't tell you any more than what it already says in the paper, the key detail that you're looking for is section 4.1.2:
""" Maglev originally used the Linux kernel network stack for packet processing. It had to interact with the NIC using kernel sockets, which brought significant overhead to packet processing including hardware and software interrupts, context switches and system calls [26]. Each packet also had to be copied from kernel to userspace and back again, which incurred additional overhead. Maglev does not require a TCP/IP stack, but only needs to find a proper backend for each packet and encapsulate it using GRE. Therefore we lost no functionality and greatly improved performance when we introduced the kernel bypass mechanism – the throughput of each Maglev machine is improved by more than a factor of five. """
My understanding is that RPS just steers packets into the Linux TCP stack. Nothing on the maglev machine wants to process the TCP layer; that workload can be distributed over the backends.