News for hackers. Stuff that matters.
Re:Serving Netflix Video at 400Gb/s on FreeBSD [pdf] by betaby on Sunday September 19, 2021 @11:25AM (#28585041)
How exactly 'Constrained to use 1 IP address per host' helps eliminate cross-NUMA transfers?
Re:Serving Netflix Video at 400Gb/s on FreeBSD [pdf] by drewg123 on Sunday September 19, 2021 @11:29AM (#28585083)
If we could use more than 1 IP, then we could treat 1 400Gb box as 4 100Gb boxes. That could lead to the "perfect case" every time, since connections would always stay local to the numa node where content is present.
Re:Serving Netflix Video at 400Gb/s on FreeBSD [pdf] by toast0 on Sunday September 19, 2021 @08:36PM (#28589279)
I wonder if you could steer clients away from connections where the NIC and storage nodes are mismatched.

Something like close the connection after N requests/N minutes if the nodes are mismatched, but leave it open indefinitely if they match.

There's of course a lot of ways for that to not be very helpful. You'd still have only a 25% of getting a port number that hashes to the right node the next time (assuming tcp port number is involved at all, if it's just src and dest ips then client connections from the same IP would always hash the same, and that's probably a big portion of your clients), and if establishing connections is expensive enough (or clients aren't good at it) then that's a negative. Also if a stream's files don't tend to stay on the same node, then churning connections to get to the right node doesn't help if the next segment is on a different node. I'm sure there are other scenarios too.

I know some other CDN appliance setups do use multiple IPs, so you probably could get more, but it would add administrative stress.

Re:Serving Netflix Video at 400Gb/s on FreeBSD [pdf] by topranks on Monday September 20, 2021 @03:51AM (#28591259)
Is there a reason you can’t?

In a past life we broke LAGs up to use different subnets per port to prevent traffic crossing the NUMA bridge.

I’m sure there are good reasons you didn’t take this approach, be interesting to hear them.

Re:Serving Netflix Video at 400Gb/s on FreeBSD [pdf] by betaby on Sunday September 19, 2021 @11:33AM (#28585116)
There is no kTLS for IPv6? IPv6 space is abundant and most mobiles is USA/Canada have IPv6. Won't that solve the problem?
Re:Serving Netflix Video at 400Gb/s on FreeBSD [pdf] by drewg123 on Sunday September 19, 2021 @11:36AM (#28585142)
IPv4 and IPv6 can both use kTLS. We offer service via V6, but most clients connect via IPv4. It differs by region, and even time of day, but IPv4 is still the vast majority of traffic.
Re:Serving Netflix Video at 400Gb/s on FreeBSD [pdf] by chungy on Sunday September 19, 2021 @12:29PM (#28585601)
I've had to blackhole Netflix IPv6 ranges on my router because Netflix would identify my IPv6 connection as being a "VPN" even though it's not.
Re:Serving Netflix Video at 400Gb/s on FreeBSD [pdf] by drewg123 on Sunday September 19, 2021 @12:45PM (#28585743)
If you could email me your ranges, I can try to look into it internally. Use my last name at gmail.com (or at freebsd.org)
Re:Serving Netflix Video at 400Gb/s on FreeBSD [pdf] by genewitch on Sunday September 19, 2021 @02:55PM (#28586887)
I'm not GP; Hurricane Electric ipv6 always had Netflix think I was on a VPN, but now I have real ipv6 through the same ISP, I just pay more money so Netflix doesn't complain anymore.
Re:Serving Netflix Video at 400Gb/s on FreeBSD [pdf] by 0x0000000 on Monday September 20, 2021 @12:21PM (#28595503)
Are you referring to HE's tunnel broker service?

If so, then yeah, that's a VPN.