News for hackers. Stuff that matters.
Re:NUMA Siloing in the FreeBSD Network Stack [pdf] by mosselman on Sunday September 22, 2019 @04:59PM (#21043899)
Isn't it great that we work in an industry where a company like Netflix, which has increasing competition lately, just shares potential competitive advantages like this?

I get that, in part, this is a way to get people excited to work for Netflix, however, the people working on this are, probably, pretty proud of what they achieved and thus like to share it with us, their colleagues at other places of work and they have the freedom to do so because of our industry's great track record of knowledge sharing.

Re:NUMA Siloing in the FreeBSD Network Stack [pdf] by throw0101a on Sunday September 22, 2019 @08:17PM (#21044932)
> ... just shares potential competitive advantages like this?

Is it really a competitive advantage though? I would think that their brand and content library are the main ones. A CDN server doesn't really count IMHO: other streaming services can create their own appliances, though I'm guessing they'll use Linux due to its ubiquity.

Though as a long-time FreeBSD fan and part-time user, I'm happy for the patches. I run a large Isilon OneFS cluster, and so hopefully these changes will help in NFS performance. :)

There is another aspect to this contribution: the FreeBSD kernel code is ever-changing, so unless Netflix wants to carry forward this patch internally into the future, it is prudent to get it committed upstream so it can be cared for by the community at-large.

Re:NUMA Siloing in the FreeBSD Network Stack [pdf] by toast0 on Sunday September 22, 2019 @11:42PM (#21045758)
> Is it really a competitive advantage though?

It totally is. These things get installed into ISP racks. ISPs have limited space and power for these things. One box pushing 200Gbps in the same space is twice as good (although, note they have 4x100Gbps NICs in this generation, there's room for more perf here).

Reducing the number of nodes helps with management of the network as well.

Re:NUMA Siloing in the FreeBSD Network Stack [pdf] by mlyle on Monday September 23, 2019 @12:11AM (#21045859)
It's funny to think that pushing 196gbps is possibly just as little as ~8000 simultaneous streams.

Also a shame that it is TLS'd. The content already has its own DRM/encryption.

Re:NUMA Siloing in the FreeBSD Network Stack [pdf] by nine_k on Monday September 23, 2019 @12:31AM (#21045921)
TLS plays a different role. It makes sure you're receiving the Netflix's DRMed and encrypted content, and not some MITMed content that also happens to be correctly DRMed and encrypted, but contains something else.
Re:NUMA Siloing in the FreeBSD Network Stack [pdf] by mlyle on Monday September 23, 2019 @01:00AM (#21046031)
Easy enough to make the content authenticate itself, instead of repeating cryptographic work for each viewer.

E.g. every 64k block contains a SHA256 hash of the next 64k block. When a user seeks, you provide the hash of the first block they'll be receiving over a trusted channel. (It already can exist at start of stream RSA signed or whatever).

But that's probably a pain when you have so many different devices to support.

(It also makes it harder to figure out what content people are watching... Though I think they use variable bitrate encoding which will fall to traffic analysis).

Re:NUMA Siloing in the FreeBSD Network Stack [pdf] by im3w1l on Monday September 23, 2019 @05:14AM (#21047205)
This exists in a more advanced form already

https://wicg.github.io/webpackage/draft-yasskin-http-origin-...

Re:NUMA Siloing in the FreeBSD Network Stack [pdf] by mlyle on Monday September 23, 2019 @12:17PM (#21050317)
No... it's not the same as signed HTTP exchanges, which you'd generally use inside TLS, and involves signing all of the data in an exchange. It wouldn't work very well for streaming, either.

The important feature of the scheme that I mentioned is that there's no per-client encryption needed, except maybe a small operation on each client seek in the stream. In general, you just serve the file, and the file contents are self-authenticating against tampering.

You might use signed HTTP exchanges to push the first hash in the chain on a seek, I guess. Or just run it over TLS. Or just have a 64k block pre-signed at the beginning stream with the hashes of 2k chosen seek points that the client can store.

Re:NUMA Siloing in the FreeBSD Network Stack [pdf] by emaste on Wednesday September 25, 2019 @09:16AM (#21070439)
TLS keeps your viewing history private from third parties (e.g. your ISP).
Re:NUMA Siloing in the FreeBSD Network Stack [pdf] by yardstick on Monday September 23, 2019 @02:00AM (#21046299)
What’s the problem with TLS? The performance overhead of it is minimal these days.
Re:NUMA Siloing in the FreeBSD Network Stack [pdf] by toast0 on Monday September 23, 2019 @02:11AM (#21046351)
It depends on the application; but for bulk transit of already encrypted (presumably by DRM) data, it is a significant overhead for little value.

You can go back and look at Netflix presentations from when they switched to TLS, and they were no longer able to saturate their network cards (and here, they're at 50% on network still)

Re:NUMA Siloing in the FreeBSD Network Stack [pdf] by throw0101a on Monday September 23, 2019 @08:36AM (#21048300)
Yes, but the parent was talking sharing the competitive advantage of the patch.

But (a) what other streaming services even have CDN appliances, and (b) of the ones that do, which are using FreeBSD as a base?

If none of Netflix's competitors will use this patch, then it makes no material difference whether they share it or not.

Re:NUMA Siloing in the FreeBSD Network Stack [pdf] by toast0 on Monday September 23, 2019 @11:19AM (#21049707)
A) at least FB and Google, probably Amazon too, plus you know, lots of CDNs have CDN appliances.

B) as far as I'm aware, everyone else is running Linux, except maybe one of the CDNs, but I don't know if that CDN is actually running appliances.

However, everyone and their uncle are launching a streaming service in the next 18 months, some of those might get enough traffic to justify appliances, and they can take Netflix's work here to get a competitive box off the bat.

Also, at any point, one of the current big guys can say "hey, netflix gets 200gbps from one box, why do we have 4? Why don't we try FreeBSD on some of these"

CDN appliances are pretty isolated, so it doesn't matter as much if most of their proprietary stack isn't ported. Chances are, they've got someone on staff with FreeBSD experience, and FreeBSD experience ages better, because there's less churn.

Re:NUMA Siloing in the FreeBSD Network Stack [pdf] by xemdetia on Monday September 23, 2019 @12:00PM (#21050122)
I would say that has less to do with Netflix and more the recent generation of Intel/EPYC server computing. These numbers seem pretty standard for this generation of Skylake/Cascade Lake and one NIC per NUMA also feels pretty standard to me. The patches also seem to be the usual problem when you are starting to care about NUMA, where there are a couple places where it just wasn't added but you need it.
Re:NUMA Siloing in the FreeBSD Network Stack [pdf] by i_cant_speel on Sunday September 22, 2019 @05:05PM (#21043942)
Agreed.

I believe technology as a whole is light-years ahead of where it would be if companies treated all of their findings and system designs as closely-guarded secrets. I am relatively early in my career and I get to open-source a project that I have been working on for the first time and it feels great.

Re:NUMA Siloing in the FreeBSD Network Stack [pdf] by um_ya on Sunday September 22, 2019 @05:31PM (#21044083)
Just imagine how far ahead the other industries would be if we handled patent and trademark law like we do in the software industry.
Re:NUMA Siloing in the FreeBSD Network Stack [pdf] by Matthias247 on Sunday September 22, 2019 @05:38PM (#21044116)
I am actually not too sure whether it matters. The amount of people that have the skills to implement such a solution is not that high. There are are only so many people who can do high performance kernel, encryption and networking work.

They might give the advantages away if they release everything, so that others can simply deploy it or offer it as a hosted solution. But the pure documentation what they did are more like advertisements for interesting engineer work @ Netflix.

Re:NUMA Siloing in the FreeBSD Network Stack [pdf] by secretasiandan on Sunday September 22, 2019 @08:26PM (#21044985)
I like it, I think its spreading, and I think its a good thing for the world.

"Top U.S. CEOs say companies should put social responsibility above profit" https://www.reuters.com/article/us-jp-morgan-business-roundt...

Re:NUMA Siloing in the FreeBSD Network Stack [pdf] by im3w1l on Monday September 23, 2019 @05:19AM (#21047223)
The part that worries me is that companies are increasingly seeing it as their social responsibility to gang up and destroy "bad" people like Stallman.
Re:NUMA Siloing in the FreeBSD Network Stack [pdf] by mschuster91 on Monday September 23, 2019 @09:52AM (#21048924)
Stallman has always been a neckbeard and either the industry grows up and shows some responsibility (which means kicking out or massively restrain the neckbeards) or it's going to be held liable itself for their actions.
Re:NUMA Siloing in the FreeBSD Network Stack [pdf] by 693471 on Tuesday September 24, 2019 @07:40AM (#21058856)
Social responsibility over profit is not legal in the USA. Fiduciary duty, EBay v Newmark
Re:NUMA Siloing in the FreeBSD Network Stack [pdf] by bscphil on Sunday September 22, 2019 @05:07PM (#21043966)
> I get that, in part, this is a way to get people excited to work for Netflix

It's even more amazing to me that this is a successful tactic in light of the fact that there's no shortage of people willing to work for ad-tech firms. If working for a company with good values was such a draw, you'd think it would give them more of a competitive advantage. Maybe the real reason is that Netflix's "good values" are genuine, not just a draw for employees. I'm really not sure.

Re:NUMA Siloing in the FreeBSD Network Stack [pdf] by gigatexal on Sunday September 22, 2019 @05:13PM (#21043999)
The whole ad tech space as a data nerd is super enticing. I could work on some really cool data warehouse / database issues but at the end of the day I’d be helping an ad tech company...

Worse yet is I’ve met some really cool people in the space but it is still ad tech

Re:NUMA Siloing in the FreeBSD Network Stack [pdf] by the8472 on Sunday September 22, 2019 @05:21PM (#21044040)
I don't see how netflix' "good values" are so much better than adtech. They're not exactly saving the planet, feeding the starving or growing artificial organs. Adtech is about getting eyeballs and converting that into money. Netflix is about getting eyeballs for hours and people paying for that privilege. Contributing to OSS projects they use is sort of the "not eating babies" baseline, even oracle and google do it!
Re:NUMA Siloing in the FreeBSD Network Stack [pdf] by throwaway2048 on Sunday September 22, 2019 @06:02PM (#21044255)
Netflix doesn't involve stalking people online and then selling the data you derive to the highest bidder.
Re:NUMA Siloing in the FreeBSD Network Stack [pdf] by 29athrowaway on Sunday September 22, 2019 @05:30PM (#21044078)
They also have their own CDN, to reduce load on ISPs.

https://media.netflix.com/en/company-blog/how-netflix-works-...

Re:NUMA Siloing in the FreeBSD Network Stack [pdf] by loeg on Sunday September 22, 2019 @07:48PM (#21044765)
Specifically, the CDN part is where Netflix uses FreeBSD. Their control plane runs on Linux.
Re:NUMA Siloing in the FreeBSD Network Stack [pdf] by techslave on Sunday September 22, 2019 @08:13PM (#21044917)
you make it sound altruistic. it’s to avoid paying ISPs
Re:NUMA Siloing in the FreeBSD Network Stack [pdf] by tw04 on Sunday September 22, 2019 @09:53PM (#21045325)
It's to avoid paying last mile ISPs who have turned their monopolies into a mafia-like shakedown scheme.
Re:NUMA Siloing in the FreeBSD Network Stack [pdf] by mlyle on Monday September 23, 2019 @12:09AM (#21045850)
Those last mile ISPs have to install one of these boxes to see a benefit.

Netflix still has to work things out and get the last mile ISPs to play ball ($$$).

But this potentially saves Netflix a whole ton on transit.

Re:NUMA Siloing in the FreeBSD Network Stack [pdf] by ksec on Sunday September 22, 2019 @06:01PM (#21044253)
>just shares potential competitive advantages like this?

It is abit like Facebook Sharing their work on Servers Hardware and Database improvements. Ultimately they dont think any of those technology are their completive advantage , their core business is Social ( and Ad Tracking, but that is another topic ).

Similar to Netflix, and would be the same as Disney. Their core business is not in the delivery, but the content.

Having said that it is still very nice of them to share their work. FreeBSD, Ruby Rails, Chaos Engineering, there are lots of things I love about their Tech Stacks.

Re:NUMA Siloing in the FreeBSD Network Stack [pdf] by amelius on Sunday September 22, 2019 @06:33PM (#21044405)
Perhaps it's just because researchers need to publish in order to, you know, build their careers.
Re:NUMA Siloing in the FreeBSD Network Stack [pdf] by robbyt on Sunday September 22, 2019 @05:27PM (#21044066)
Realistically, Disney, or whomever competition, is not going to have enough technical expertise to deploy FreeBSD.
Re:NUMA Siloing in the FreeBSD Network Stack [pdf] by cwyers on Sunday September 22, 2019 @05:50PM (#21044199)
The BSD logo was created by one of Pixar's founders[1], and Disney now owns Pixar. Disney also has money. I'm sure deploying FreeBSD is well within their reach.

1) https://www.freebsd.org/copyright/daemon.html

Re:NUMA Siloing in the FreeBSD Network Stack [pdf] by cthalupa on Sunday September 22, 2019 @05:54PM (#21044217)
... You don't think Disney, a company with a revenue of roughly 60 billion dollars, can hire people with enough technical expertise to deploy FreeBSD?
Re:NUMA Siloing in the FreeBSD Network Stack [pdf] by rayiner on Sunday September 22, 2019 @06:15PM (#21044333)
Disney’s animation studio is based on Linux infrastructure: https://www.velvetjobs.com/job-posting/sr-systems-engineer-l...
Re:NUMA Siloing in the FreeBSD Network Stack [pdf] by o-__-o on Sunday September 22, 2019 @08:34PM (#21045019)
I trained the entire Disney offshore replacement team. They understand FreeBSD very well.
Re:NUMA Siloing in the FreeBSD Network Stack [pdf] by m11r on Monday September 23, 2019 @11:43AM (#21049962)
Out of curiosity, does this mean that they’re using (or have used) FreeBSD in their infrastructure or for some other purpose, or just a high level of Unix competence in that team? Any interesting details that you’re able to share?