News for hackers. Stuff that matters.
Re:Serving Netflix Video Traffic at 800Gb/s and Beyond [pdf] by Sytten on Friday August 19, 2022 @09:24AM (#32520655)
I think a fair criticism would be how many engineers they have compared to their competitors. Disney+ is on a similar scale, can they do the same/similar job with less people? And considering netflix pays top of market, how much does Disney spends for their engineering effort to get their result. Would netflix benefit from just throwing more hardware at the problem vs paying more engineers 400-500k/y to optimize?
Re:Serving Netflix Video Traffic at 800Gb/s and Beyond [pdf] by iamricks on Friday August 19, 2022 @09:42AM (#32520831)
Standing on the shoulders of giants, Netflix engineers didn't have blog posts from other companies on how to handle the scale they started facing. Facebook didn't have blog posts to reference when they scaled to 1B users. They pay for talent that have built systems that had not been built before and they have seen a return on it so they continue to do it.
Re:Serving Netflix Video Traffic at 800Gb/s and Beyond [pdf] by wowokay on Friday August 19, 2022 @12:29PM (#32523090)
Hulu was around before netflix
Re:Serving Netflix Video Traffic at 800Gb/s and Beyond [pdf] by Cerium on Friday August 19, 2022 @12:41PM (#32523255)
Sure? "After an early beta test in Oct. of that year, Hulu was made available to the public on March 12, 2008—a year after Netflix launched its own streaming service."

[1] https://www.foxbusiness.com/technology/5-things-to-know-abou...

Re:Serving Netflix Video Traffic at 800Gb/s and Beyond [pdf] by msh on Friday August 19, 2022 @12:41PM (#32523251)
Hulu was never Netflix scale. YouTube is a better example.
Re:Serving Netflix Video Traffic at 800Gb/s and Beyond [pdf] by jedberg on Friday August 19, 2022 @01:34PM (#32523886)
Youtube is very different than Netflix from a technical problem perspective. They serve free videos to anyone around the world that are uploaded by users.

It's closer to a live streaming problem than pre-encoded video like Netflix.

Having worked at Netflix I can say that the YouTube problem is much more complex.

Re:Serving Netflix Video Traffic at 800Gb/s and Beyond [pdf] by why_only_15 on Friday August 19, 2022 @04:15PM (#32525491)
I wonder what portion of Youtube's request traffic can be served with cache servers at the edge with a few hundred terabytes of storage. There's a very long tail but i would guess a significant portion of their traffic is the top ~10,000 videos at any given moment.
Re:Serving Netflix Video Traffic at 800Gb/s and Beyond [pdf] by spockz on Friday August 19, 2022 @04:42PM (#32525699)
There was a Google organised hackathon on this topic. Given a set of resources, locations, and (estimated) popularity, Optimise for video load time by determining what should be moved to the cache when and where.
Re:Serving Netflix Video Traffic at 800Gb/s and Beyond [pdf] by skyde on Saturday August 20, 2022 @01:50AM (#32529496)
did anything from the hackathon turn out to be used?
Re:Serving Netflix Video Traffic at 800Gb/s and Beyond [pdf] by spockz on Saturday August 20, 2022 @04:09AM (#32530241)
I’m not sure as I don’t work at Google and didn’t see any follow up. Maybe some in the winning teams would know.

I do think there is some temporal logic already present in Google’s algorithm which wasn’t part of the challenge.

Re:Serving Netflix Video Traffic at 800Gb/s and Beyond [pdf] by birdyrooster on Friday August 19, 2022 @01:09PM (#32523601)
Not even close. YouTube has orders of magnitude more content and vastly more users. Google Global Cache was the inspiration for Open Connect.
Re:Serving Netflix Video Traffic at 800Gb/s and Beyond [pdf] by gavin_gee on Friday August 19, 2022 @12:44PM (#32523301)
yeah and have you see the awful performance of Hulu? its basically unusable. poster child for under investing in the streaming platform.
Re:Serving Netflix Video Traffic at 800Gb/s and Beyond [pdf] by paxys on Friday August 19, 2022 @02:00PM (#32524205)
Huh? Netflix predates Hulu by over a decade.
Re:Serving Netflix Video Traffic at 800Gb/s and Beyond [pdf] by toast0 on Friday August 19, 2022 @11:25AM (#32522164)
> Would netflix benefit from just throwing more hardware at the problem vs paying more engineers 400-500k/y to optimize?

Where the CDN boxes go, you can't always just throw more hardware. There's a limited amount of space, it's not controlled by Netflix, and other people want to throw hardware into that same space. Pushing 800gbps in the same amount of space that others do 80gbps (or less) is a big deal.

Re:Serving Netflix Video Traffic at 800Gb/s and Beyond [pdf] by pclmulqdq on Friday August 19, 2022 @11:22AM (#32522111)
The engineers are definitely cost-effective at this scale. They may be the highest-leverage engineers at the company in terms of $ earned from their efforts compared to $ spent. The improvements that come from performance engineers at large companies are frequently worth $10M/year/person or more.

Most companies maintain internal calculations of these sorts of things, and make rational decisions.

Re:Serving Netflix Video Traffic at 800Gb/s and Beyond [pdf] by gregsadetsky on Friday August 19, 2022 @12:21PM (#32522983)
Sorry for the tangent, but really curious to ask:

When you say that companies maintain internal calculations of the benefits, would you say that it’s (extremely roughly) something like: $10M benefit, need 5 core engineers + benefits + PM + testing lab etc etc -> we can spend up to $500k per eng give or take.

Or is the $10M one number (that would be held somewhat secretly internally at the company) and the salaries mostly represents where the market is? Does the (salary) market take into account the down-the-line $10M value?

Basically, could those engs negotiate to be paid more, or are they already sort of paid close to exactly what the group they’re part of generates in terms of revenue?

Thanks!

—

I see that you said $10M per person, not for the “network optimization group”. Hmm. So it would be fair to say that the engs are definitely not paid according to the value they generate..? I wouldn’t be surprised by that but just to confirm.

Re:Serving Netflix Video Traffic at 800Gb/s and Beyond [pdf] by donavanm on Friday August 19, 2022 @12:52PM (#32523412)
If you are an employee there is little to no relationship between your output and your compensation. Employer employee relationships are based on the cost to the employer to secure equivalent or better output.

Secondly, yes $10M per employee of revenue or cash flow is pretty reasonable for similar companies. The prioritization is NOT “how many employees per $MM.” The allocation is “what opportunity is the highest $MM return per available employee.”

Re:Serving Netflix Video Traffic at 800Gb/s and Beyond [pdf] by closeparen on Friday August 19, 2022 @07:36PM (#32527410)
Relatively few companies have the scale or the efficiency for it to be profitable to hire engineers at Silicon Valley wages.
Re:Serving Netflix Video Traffic at 800Gb/s and Beyond [pdf] by donavanm on Friday August 19, 2022 @08:10PM (#32527627)
Sure. Lots of companies and employees are located elsewhere. And software + services is a heck of a financial drug. I dont think that changes the underlying message of how to think about employee comp or business opportunities.
Re:Serving Netflix Video Traffic at 800Gb/s and Beyond [pdf] by closeparen on Friday August 19, 2022 @08:37PM (#32527816)
Compensation is negotiated up and down marginally by the cost of hiring someone else, but bounded above by profitability. You could be the best price:performance employee in the world for a small business, there's just not all that much they could pay you. A huge mega-profitable tech company can pay whatever it takes for top talent and also get away with sometimes inefficiently overpaying rest-and-vesters.
Re:Serving Netflix Video Traffic at 800Gb/s and Beyond [pdf] by Hermitian909 on Friday August 19, 2022 @12:33PM (#32523140)
Not OP, but 1 engineer -> 10M of benefit sounds right for my company.

In terms of negotiation, it really depends on how differentiated your skills are. Short answer is that if you can convince management that it would be difficult to find other engineers who could deliver the optimizations you're delivering, yes, you have leverage.

Re:Serving Netflix Video Traffic at 800Gb/s and Beyond [pdf] by pclmulqdq on Friday August 19, 2022 @01:14PM (#32523661)
This is exactly right about negotiation and your skillset. I have seen performance engineers in the right place at the right time get 10-20% of their benefit to the company (I have seen both $1 million/year compensation for line workers and $10+ million/year for very senior folks).

Very highly skilled engineers in specific niches can basically price themselves like monopolists, because the company can easily figure out how much money they are leaving on the table by not hiring them. This is not like "feature work" engineers, whose value is very nebulous and unknown.

Re:Serving Netflix Video Traffic at 800Gb/s and Beyond [pdf] by pclmulqdq on Friday August 19, 2022 @01:04PM (#32523551)
The simple fact is that you are not paid for the value you create. You are paid based on the salary you can demand. For performance engineers, $10 million/year/person opportunities are kind of rare, meaning that you can't demand close to that. Your alternatives to big tech are things like wall street, which pay very well, so you can demand a higher salary (and/or higher level) than a normal engineer of your skill would get. However, this is nowhere near the value of the work.
Re:Serving Netflix Video Traffic at 800Gb/s and Beyond [pdf] by paxys on Friday August 19, 2022 @09:30AM (#32520704)
Disney (the company) has 20x the number of employees as Netflix, and just 2x the market cap (in fact they were briefly worth the same last year), ~2x the revenue and 2/5 the net income. So Netflix is clearly doing something right.
Re:Serving Netflix Video Traffic at 800Gb/s and Beyond [pdf] by ziddoap on Friday August 19, 2022 @09:35AM (#32520751)
>Disney (the company) has 20x the number of employees

Is that all of Disney or just Disney+?

It doesn't seem like that would be a useful statistic if that includes completely unrelated positions (e.g. does that 20x statistic include Disney employees working at Disney Land/World serving up hotdogs? Because they probably don't contribute much to the streaming service)

Re:Serving Netflix Video Traffic at 800Gb/s and Beyond [pdf] by briffle on Friday August 19, 2022 @10:10AM (#32521143)
Netflix also has production studios they now own making content.
Re:Serving Netflix Video Traffic at 800Gb/s and Beyond [pdf] by thfuran on Friday August 19, 2022 @11:58AM (#32522635)
Content like hotdogs at an amusement park?
Re:Serving Netflix Video Traffic at 800Gb/s and Beyond [pdf] by diab0lic on Friday August 19, 2022 @03:06PM (#32524884)
Re:Serving Netflix Video Traffic at 800Gb/s and Beyond [pdf] by scrlk on Friday August 19, 2022 @10:33AM (#32521456)
Disney Streaming had 850 employees as of 2017 [0] (can't find any newer figures); LinkedIn is suggesting 1k-5k.

[0] https://en.wikipedia.org/wiki/Disney_Streaming

Re:Serving Netflix Video Traffic at 800Gb/s and Beyond [pdf] by eru on Friday August 19, 2022 @10:37AM (#32521518)
Perhaps they are just running different business models?

Walmart's market cap per employee is probably much, much lower than Disney or Netflix, too. That doesn't mean Walmart is doing anything wrong.

Re:Serving Netflix Video Traffic at 800Gb/s and Beyond [pdf] by jwmoz on Friday August 19, 2022 @10:04AM (#32521070)
I watch Disney content sometimes and it constantly drops or freezes, you can see the difference in quality compared to Netflix.
Re:Serving Netflix Video Traffic at 800Gb/s and Beyond [pdf] by bmurphy1976 on Friday August 19, 2022 @10:58AM (#32521823)
Yeah, you can totally see the difference. Netflix encoding looks like shit.

I've done a lot of video processing professionally (the server side stuff, exactly what Netflix does) and Netflix is by far the worst of all the streaming providers. They absolutely sacrifice the quality of the video to save bandwidth costs in aggregate and it shows (or more accurately it doesn't show, all the fidelity is lost).

Re:Serving Netflix Video Traffic at 800Gb/s and Beyond [pdf] by bagels on Friday August 19, 2022 @12:32PM (#32523128)
Your isp may be throttling bandwidth for Netflix, leading to lower quality encodings being served to you. Comcast does this, for instance.
Re:Serving Netflix Video Traffic at 800Gb/s and Beyond [pdf] by jedberg on Friday August 19, 2022 @02:18PM (#32524398)
Are you getting the best Netflix encodings? You might be getting worse quality because your ISP throttles Netflix.
Re:Serving Netflix Video Traffic at 800Gb/s and Beyond [pdf] by bmurphy1976 on Monday August 22, 2022 @01:59PM (#32554932)
For anybody looking at this after-the-fact, I'm not making it up. Netflix really does look like shit: https://news.ycombinator.com/item?id=32553958
Re:Serving Netflix Video Traffic at 800Gb/s and Beyond [pdf] by mkmk on Friday August 19, 2022 @12:06PM (#32522758)
Do you think it's worth it to pay the extra $5-10/month for premium quality? https://help.netflix.com/en/node/24926
Re:Serving Netflix Video Traffic at 800Gb/s and Beyond [pdf] by bradstewart on Friday August 19, 2022 @03:33PM (#32525138)
Even the Premium 4k streams have surprisingly low bitrates and, occasionally, framerates. I dug out the blu-ray player the other day and was absolutely shocked how good things looked and, even more so, sounded--the audio quality from Netflix (and most streaming services, really) is simply atrocious.
Re:Serving Netflix Video Traffic at 800Gb/s and Beyond [pdf] by LtdJorge on Saturday August 20, 2022 @01:13AM (#32529289)
Of course, but you can't expect 100GB movies for 4K HDR from an online provider.
Re:Serving Netflix Video Traffic at 800Gb/s and Beyond [pdf] by Mindwipe on Saturday August 20, 2022 @08:59PM (#32536480)
The Sony Bravia Core service isn't far off that to be honest.
Re:Serving Netflix Video Traffic at 800Gb/s and Beyond [pdf] by nicce on Friday August 19, 2022 @10:13AM (#32521189)
You can’t make such conclusions from your own experience. It is one form of bias. There are many variables. For me it is the opposite, for example.
Re:Serving Netflix Video Traffic at 800Gb/s and Beyond [pdf] by entropie on Friday August 19, 2022 @09:28AM (#32520694)
I wasn't able to watch disney+ via chromecast for like a year in 4k. Stuttering every 10 seconds or so. I never had problems like this with netflix.
Re:Serving Netflix Video Traffic at 800Gb/s and Beyond [pdf] by criddell on Friday August 19, 2022 @09:56AM (#32520980)
I guess you weren't a Comcast customer in 2014 trying to watch Netflix and getting low quality, stuttering video. At the time lots of people tried to frame it as a net neutrality issue but in the end I think it was a peering dispute that involved a third party.

https://www.wsj.com/articles/SB10001424052702304834704579401...

Re:Serving Netflix Video Traffic at 800Gb/s and Beyond [pdf] by loopercal on Friday August 19, 2022 @10:08AM (#32521123)
I think this just validates their points. Netflix has more engineers and 8 years of them building and fixing things, so they have fewer issues.
Re:Serving Netflix Video Traffic at 800Gb/s and Beyond [pdf] by slillibri on Friday August 19, 2022 @11:27AM (#32522183)
Disney bought a majority ownership in BAMTECH to build Disney+.
Re:Serving Netflix Video Traffic at 800Gb/s and Beyond [pdf] by rybosworld on Friday August 19, 2022 @09:35AM (#32520758)
That seems like a fair point if you just consider the video streaming. I know that Netflix wants to break into gaming. I'd imagine the bandwidth required for that is higher than streaming videos.
Re:Serving Netflix Video Traffic at 800Gb/s and Beyond [pdf] by jon-wood on Friday August 19, 2022 @10:21AM (#32521305)
It's really not, especially if you look at their current model for doing so. Netflix at the moment are breaking into mobile gaming, which means the bandwidth requirements are placed on Apple/Google's app store infrastructure. I'd be surprised if Netflix don't have any sort of metrics gathering infrastructure to judge how much people are playing those games, but they're also likely reusing the same infrastructure used by Netflix video streaming for that, so the incremental increase in load may well be negligible.
Re:Serving Netflix Video Traffic at 800Gb/s and Beyond [pdf] by rybosworld on Friday August 19, 2022 @01:00PM (#32523508)
I was referring to their plans for a game streaming service.