News for hackers. Stuff that matters.
Re:10M Concurrent Websockets by eloff on Saturday March 19, 2016 @04:35PM (#11320306)
This is kind of lousy actually to use that much memory and other resources for 10M nearly completely idle connections.

If the kernel is bypassed, I have reason to believe Go can handle the line rate of 16.7M packets/sec(on 10gbps nic.) I would love to integrate Intel's DPDK with Go. I've done some ground work on this before - it's very feasible to just replace the Go network stack in a transparent way. If any company is interested in sponsoring that as an open-source project, my contact info is in my profile.

Re:10M Concurrent Websockets by jeff_marshall on Saturday March 19, 2016 @10:12PM (#11321461)
As I'm sure you are well aware, there is quite a large gap between the DPDK and a working TCP stack.

Do you plan to write the TCP stack from scratch? If not, are you planning to leverage one of the golang unikernel back-end ports?

10Gb/s (even per-core) doesn't seem like much of a selling point for a network stack on it's own, given where the rest of the industry is.

Re:10M Concurrent Websockets by eloff on Sunday March 20, 2016 @12:04AM (#11321746)
I wouldn't dare write my own. I'm not expert enough in TCP/IP stacks to know what I don't know that will hurt me. I was thinking of lifting the relatively simple TCP/IP stack from the Apache licensed Seastar framework. Since that also is built for DPDK and sees production use I know that it should be easy to adapt and probably doesn't have serious flaws.

I'm actually strongly tempted to just to glue Seastar and Go together. That's the lowest risk approach.

Re:10M Concurrent Websockets by hadagribble on Sunday March 20, 2016 @12:23AM (#11321795)
I haven't gotten around to trying it myself yet, but OpenFastpath+OpenDataplane supposedly has a userspace port of the FreeBSD stack that runs on DPDK through ODP-DPDK.

http://www.openfastpath.org/

http://www.opendataplane.org/

Re:10M Concurrent Websockets by jeff_marshall on Sunday March 20, 2016 @04:36PM (#11324534)
Thanks for the pointer to Seastar, I wasn't aware of that project.

Good luck with your work - I'm curious to see the result.

Re:10M Concurrent Websockets by omgtehlion on Saturday March 19, 2016 @08:18PM (#11321165)
You can easily throw some openonload/vma at it before going full DPDK
Re:10M Concurrent Websockets by eloff on Saturday March 19, 2016 @08:56PM (#11321262)
Yes openonload is an easy way to get a boost. It doesn't help you much in the commodity cloud though. There are still a lot of advantages to managing your own hardware when it's feasible.