News for hackers. Stuff that matters.
Re:PfSense – WireGuard Returns as Experimental Package by carlhjerpe on Sunday May 09, 2021 @10:10AM (#27096026)
DPDK is fast in userspace by taking ownership of devices (I believe) so that there's no context switching anyways.
Re:PfSense – WireGuard Returns as Experimental Package by SigmundA on Sunday May 09, 2021 @10:43AM (#27096287)
My understanding is DPDK has much smaller set a supported nic's since it doesn't use the kernel drivers for the card along with all the other things that the kernel provides.

Wireguard in kernel gives great performance while just working if your kernel is already working on the hardware. Wireguard is so small it is considered easy to audit so is a prime candidate for kernel implementation.

DPDK is listed as an "Exotic" todo for Wireguard:

https://www.wireguard.com/todo/

Re:PfSense – WireGuard Returns as Experimental Package by gonzo on Sunday May 09, 2021 @11:49AM (#27096960)
There is already an implementation of Wireguard in VPP, which runs over DPDK.
Re:PfSense – WireGuard Returns as Experimental Package by tlamponi on Sunday May 09, 2021 @10:55AM (#27096405)
Throwing out a few points regarding DPDK/low overhead data/event exchange between kernel and userspace:

1. It is faster if you make it so, i.e., you need to invest quite some work to get to a point where it's faster than kernel - but yea, with that you can make it go brrrrrrrt in quite a few use cases, it's more like an SDK.

2. Some NICs support loading a smaller program to operate directly on the NIC, e.g., through eBPF - nothing gets faster than that as it avoids that (some) traffic hits the CPU at all. But again, it's work to create and maintain and only feasible for specialized stuff.

3. Note that with Linux[0] a lot of the context switch overhead can be avoided by leveraging the state of the art async framework IO-uring, BSD may already have or get something similarily[0]. I recommend reading the paper of the original author[0], it's fascinating IMO.

[0]: I know this is about BSD, so somewhat off-topic, but still shows that most of the context switch complexity can be avoided while not re-inventing the wheels in user-space.

[1]: https://kernel.dk/io_uring.pdf

Re:PfSense – WireGuard Returns as Experimental Package by 0x000000E2 on Sunday May 09, 2021 @03:34PM (#27098989)
I'll add that with DPDK you end up needing your own TCP stack if you're doing anything above layer 3.

TCP is a beast with opportunity for countless subtle bugs. The Linux kernel is perhaps the best implementation there is. Many userspace implementations are simplified and missing features.

Even giants like Cloudflare use Linux kernel for routing when they need to operate above layer 3. They only use DPDK for very low level features.

DPDK has its place but for most use cases you need to man handle TCP and you're better off using kernel packet handling for that

Re:PfSense – WireGuard Returns as Experimental Package by gonzo on Sunday May 09, 2021 @05:25PM (#27099938)
> use Linux kernel for routing when they need to operate above layer 3.

What could you possibly mean by this?

Re:PfSense – WireGuard Returns as Experimental Package by 0x000000E2 on Sunday May 09, 2021 @05:57PM (#27100213)
When they need to handle UPD and TCP. Cloudflare's web application firewalls run at TCP and HTTP layer and use tuned Linux kernels for processing, not DPDK and userspace TCP as some think

Their line rate stuff for flood mitigation uses DPDK or similar

Re:PfSense – WireGuard Returns as Experimental Package by gonzo on Sunday May 09, 2021 @11:48AM (#27096951)
Nothing there was DPDK, just FreeBSD kernel networking.

The Wireguard implementation in VPP runs at about 5Gbps in a simple single-stream iperf test.

Re:PfSense – WireGuard Returns as Experimental Package by SigmundA on Sunday May 09, 2021 @12:51PM (#27097506)
5Gbps on what hardware and how does that compare to the kernel implementation on the same hardware? Curious to know how big a difference DPDK makes. Also what about cpu time, I know typically DPDK takes more CPU due to polling, normally want to dedicate cpus to DPDK right?
Re:PfSense – WireGuard Returns as Experimental Package by gonzo on Sunday May 09, 2021 @05:43PM (#27100106)
Very new Xeon at 3.5GHz. 5.75gbps.

Kernel Wireguard is almost 3gbps.

But Wireguard is mostly held back by the slow crypto implementation, which is CPU-bound.

AES-GCM IPsec on the same hardware is 19.5gbps, limited by the 25gbps NIC, using again, single-stream iperf with ipsec in VPP.

DPDK only needs dedicated cores for the poll mode drivers, and these days the Intel and mellanox NICs can be run in interrupt mode.

Re:PfSense – WireGuard Returns as Experimental Package by SigmundA on Sunday May 09, 2021 @05:57PM (#27100217)
I thought interrupt mode reduced performance, that was the tradeoff, is the throughput your quoting in interrupt or polling mode?

Good numbers either way, but you again much more limited hardware support since the drivers need to be DPDK specific.

Re:PfSense – WireGuard Returns as Experimental Package by gonzo on Sunday May 09, 2021 @11:04PM (#27102276)
You can also run VPP over AF_PACKET (slower, but faster than the kernel) of AF_XDP (almost as fast as DPDK interrupt mode.)

But Wireguard is still crypto-limited, the AES-GCM result proves it.

Re:PfSense – WireGuard Returns as Experimental Package by SigmundA on Monday May 10, 2021 @08:30AM (#27105249)
Wireguard isn't doing to bad considering its not hardware accelerated like AES-GCM.

Also looks like AF_PACKET and AF_XDP are basically an alternative to DPDK with similar functionality but built into the kernel using kernel drivers, new to me.