Yes, this is absolutely critical. Atomic ops (including taking a lock) when you have a large number of cores just completely kills performance.
>Do they make nics with 128 queues now?
Yep, the Intel E810 100gbit controller supports 256 queue pairs for just one example.
Hope you can process that traffic pretty fast, the amount of memory dedicated to packet buffers with that many queues is making my head spin a bit.
My use case was just tcp proxy, inspecting the first few bytes to determine destination (http vs https vs proprietary) and add proxy protocol headers to the origin. Setup is somewhat expensive (tcp table manipulation isn't cheap), but data on established connections is simple and easy.
The bottleneck was connection establishment in the kernel, and as traffic decreased it became harder to run meaningful tests without great risk (send all the traffic to one server to see if it can handle it is fun, but unsafe), and we had to keep a fair number of servers to keep the IPs from the hosting provider, so we had way more capacity than needed and optimizing more wasn't a good use of time :(
Connection establishment in the kernel was an issue before kernel 3.8 I think though not sure. The problem is finding a spare port when you use a lot of concurrent connections and at some point it costs a lot. The "source" directive in haproxy was made to overcome this (you can define a per-server range so that the kernel doesn't have to scan its lists). But nowadays more efficient mechanisms were implemented in the kernel, that we support in haproxy (using IP_BIND_ADDRESS_NO_PORT) and we haven't witnessed these painful moments for a while now.
Somewhere my (not very great, but working) patch set is floating around, where I added rss aware port selection to HAProxy, so client and origin connections would both hash to the same NIC RX queue, and HAProxy never had to deal with cross-cpu communication for reading sockets.
I haven't looked at this code in a long time! Haven't gotten to a computer and it's hard to read this on a phone, so... And I left where I wrote it, so I can't confirm if there were any substantial changes after these. My fuzzy memory is this may be all the HAProxy changes, then I focused on kernel level changes.
Of course if HAProxy performs well enough, then that's still a million times easier.
Also, I found it best to work with everything avoiding cross CPU communication; making sure everything happens on the CPU core that handles the RX queue for the incoming connection. My NICs only did 16 queues, so more cores than that wasn't useful for me. Do they make nics with 128 queues now? (Selecting outgoing ports so traffic to/from origin stays on the same CPU is tricky, but also solves the problem of making sure there's no port contention between the processes)