Kernel bypass is not a panacea. Even with a totally userspace networking stack, there will be a lot of per-packet rather than per-byte processing, and often packets from different sources will be sufficiently interleaved that you can't even do any kind of useful batching fastpath .

taking cpu-cores out of normal scheduling, and dedicating them for just packet-forwarding etc. is the first step. not sure how popular ddio is outside of intel ecosystem, but shunting packets between nic and cpu-cache (to and from) is very useful for such workloads imho.

The organization in question already bypasses the kernel with their “SNAP” scheme. Perhaps they are motivated to keep improving kernel IP stack simply because practically all GCP customers still use it.

https://blog.acolyer.org/2019/11/11/snap-networking/

> But getting people to dedicate a core/thread [to RX/TX polling] is also a hard sell.

I think that if you're at the level where the kernel network stack is not enough, core pinning is a no brainer. You're probably spending more than one core in receiving data, moving to a polling thread pinned to a single core will not only improve performance due to the polling, but also due to the scheduler not pushing your process out and to the possibility of putting the threads on the appropriate NUMA nodes.

Or simply use network cards with full TCP offload (https://en.wikipedia.org/wiki/TCP_offload_engine).

Part of the pain with kernel bypass is memory management. For real performance gains you need zero copy and for that you need to work inside the packets. Perfectly doable for a green field project not so east I think for a large existing system.

I wanted to pay someone to work on a dpdk layer for zeromq, I'm not sure anyone would be interested but I feel it'd help alleviate part of the dpdk pain.

What's the's pain exactly with using DPDK?

You have to be root/suid, have to implement your netstack yourself (ip, tcp, etc.), have to use the special threads which may or may not play nice with other threading libraries, locking and lock-free stuff, and there are subtle differences between controler behaviours that are smoothed over by the kernel and drivers, but hey you don't want those. For very simple forwarding and L2/L3 stuff it's easy to start and you can get to high perf quick enough. For anything applicative it can quickly become hard. I lost so much time with e.g. jumbo frames support, arp, frame re-ordering and low-level footguns, honestly sometimes I miss the pain of plain sockets.

And in the end what I want is mostly giving a large reassembled 2-100MB buffer to userland to avx512-stuff all over - same as I do with zmq on a (bunch of) 10Gb link(s). Or send to gpu via gpudirect, and the joy of programming DMAs instead of 'normal' cudaMemcpyAsync. And don't forget to reserve a (bunch of) physical machines for CI.

I'm not saying it's not useful, on the contrary. You get to send/receive 800Gbit/s of traffic on a single socket system! But it feels like a setback in usability, learning curve.

I'm saying I'd like a simpler API over it, to transmit/receive pub/sub large messages.

Thanks for the wonderfully detailed response. I had a couple questions:

>"And in the end what I want is mostly giving a large reassembled 2-100MB buffer to userland to avx512-stuff all over - same as I do with zmq"

How does the avx512 come into play here? I know its mainly used for HPC type workloads. Is there a more general usage pattern for it as well? I know Linux was very vocal about his disdain for avx512 and mentioned people were using it for memcpy[1] Is that the case for DPDK?

Second is "zmq" here ZeroMQ, the messaging library?

I would type of application space you work in.

[1] https://www.techradar.com/news/linux-founder-chastises-intel...

Ah it's mostly signal/video processing, very high throughput, compute-heavy, very little branching. AVX was kind of off-topic there, sorry...

AVX512 (and vector instructions) makes sense if you have very repetitive dense operations to do on your data. Think convolutions, filters, BLAS, FFT... Even better with dpdk since they run with hugepages so no tlb thrashing.

DPDK has an option to use AVX512 for memory copy but I mostly do the computing + copy results in one pass, to maximize throughput.

Zmq for zeromq yes, sorry. The abstraction (send arbitrary sized messages, receive messages, publish, subscribe, route messages... Same API for in-process, inter-process, inter-node) is so nice I miss it when we go down some levels of abstraction, especially with large large messages.

I'm not sure I agree with Linus there. Intel is/was in a pinch, unable to ramp up FLOPS/mm2 and core count, they reached for vector instructions, in an extremely pragmatic 'what do you need them for - OK you'll get exactly those! And if you need something else, well... Have fun with the fun instruction set'.

Intel also had (understandable) frequency throttling problems with how dense/hot the fused-multiply-add could get, but still painful and not detailed (discover weird frequency quirks on your 8000USD chip yay!) and have also been (to me) very unclear on the AVX512 roadmap (see recent Alderlake adventures).

I get that Linus would like simpler, less kludgy solutions (more cores, more efficient cores, etc.) but if they exist in the cpu world, I'm waiting. Maybe ARM SVE whenever it will hit mainstream, maybe... The way for more raw flops is for the moment GPUs/accelerators - TPU, tenstorrent, FPGAs, mining ASICs being the extreme there - but they're their own cans of worms :-).

So if you want very low latency (no going back and forth through PCIe and not motivated enough to use GPUdirect...) and still in the PC world, you're stuck. But AVX512 is actually a very good instruction set! Branch registers, most operations available, cross-lane instructions, all the shuffles you can imagine (well more than I can imagine) and recently avx512-fp16, the vectorized popcount... It's all getting better and better.

A real shame we don't have more hardware with it - starting TigerLake you get it in consumer hardware! good luck with thermals but it gives a taste.