News for hackers. Stuff that matters.
Re:Verizon Throttles Netflix Subscribers in Test It Doesn't ... by mrkrabo on Monday July 24, 2017 @12:33PM (#14839880)
Why shouldn't Verizon and Google split the bill?
Re:Verizon Throttles Netflix Subscribers in Test It Doesn't ... by raisedbyninjas on Monday July 24, 2017 @12:42PM (#14839976)
In a way, they used to. Peering agreements between large networks allow unbilled traffic from either party with the understanding that both would be serving and consuming similar amounts of traffic. With the emergence of the network neutrality debate ISPs have claimed that peering agreements are not appropriate because content providers "send" more traffic than they receive. This of course ignores the fact that their customers are requesting the traffic and their whole business model is designed on asymetric traffic.
Re:Verizon Throttles Netflix Subscribers in Test It Doesn't ... by toast0 on Monday July 24, 2017 @03:30PM (#14841843)
Peering disputes have always been when the party wanting to peer had an unbalanced traffic ratio, and the other party didn't want to peer (possibly for other reasons), and they started much earlier than the network neutrality debates.

The recent changes are more around who is involved in the peering disputes. It used to be smaller ISPs trying to get peering with larger ISPs, such as Cogent trying to get peering with [name your favorite, or PSINet vs Cable and Wireless; and most often the ISP refusing to peer didn't have residential customers themselves. Now it's more often the content providers themselves trying to peer with the residential isps directly. A major factor here is the huge consolidation of residential ISPs, but also the consolidation of content providers.

Consolidation of residential ISPs means each ISP is big enough to run their own backbone, and as a result they can credibly have strict peering requirements. Smaller, regional ISPs will tend to want to peer, because otherwise the traffic will come through on paid transit connections. Large ISPs may not care; because of their size, they may not be paying anyone for transit, and because of the common asymmetric nature of residential connections, there's not likely to be many networks where the large ISP is on the wrong side of the ratio.

If I were one of these content providers, I would spend a lot more time messing with the large ISPs. Figure out how to make the traffic cost them money, so they'll want to peer. Provide transit to data backup services to try to make the ratios less unbalanced. Run campaigns suggesting that residential ISPs should be paying their customers, given that the traffic is unbalanced. Etc.

Re:Verizon Throttles Netflix Subscribers in Test It Doesn't ... by thanksgiving on Monday July 24, 2017 @04:09PM (#14842196)
> Peering disputes have always been when the party wanting to peer had an unbalanced traffic ratio, and the other party didn't want to peer (possibly for other reasons), and they started much earlier than the network neutrality debates. The recent changes are more around who is involved in the peering disputes. It used to be smaller ISPs trying to get peering with larger ISPs, such as Cogent trying to get peering with [name your favorite, or PSINet vs Cable and Wireless; and most often the ISP refusing to peer didn't have residential customers themselves. Now it's more often the content providers themselves trying to peer with the residential isps directly. A major factor here is the huge consolidation of residential ISPs, but also the consolidation of content providers.

> Consolidation of residential ISPs means each ISP is big enough to run their own backbone, and as a result they can credibly have strict peering requirements. Smaller, regional ISPs will tend to want to peer, because otherwise the traffic will come through on paid transit connections. Large ISPs may not care; because of their size, they may not be paying anyone for transit, and because of the common asymmetric nature of residential connections, there's not likely to be many networks where the large ISP is on the wrong side of the ratio.

> If I were one of these content providers, I would spend a lot more time messing with the large ISPs. Figure out how to make the traffic cost them money, so they'll want to peer. Provide transit to data backup services to try to make the ratios less unbalanced. Run campaigns suggesting that residential ISPs should be paying their customers, given that the traffic is unbalanced. Etc.

I can upload all my files to Google Drive and all my photos and videos to Google Photos if you think it helps the ratio...

Re:Verizon Throttles Netflix Subscribers in Test It Doesn't ... by jessaustin on Monday July 24, 2017 @04:35PM (#14842413)
I've long thought the Netflix app should simply upload random noise, to "balance out" the tremendous download/upload difference with which some poor ISPs struggle so.
Re:Verizon Throttles Netflix Subscribers in Test It Doesn't ... by function_seven on Monday July 24, 2017 @03:36PM (#14841902)
> This of course ignores the fact that their customers are requesting the traffic and their whole business model is designed on asymetric traffic.

Exactly. If the Internet worked on this model—where the recipient of the net imbalance was paid money for it—I'd get a check from Verizon each month instead of a bill!

Re:Verizon Throttles Netflix Subscribers in Test It Doesn't ... by jmuguy on Monday July 24, 2017 @12:39PM (#14839948)
My understanding is that its not an issue of the cost to "upgrade" the interconnection infrastructure. That cost is likely negligible. What's happening is that VZ is refusing to do it, and allowing service to degrade, unless Google pays X. So yeah, basically extortion. Google has already paid for bandwidth, and this is traffic that VZ's paying customers are requesting.
Re:Verizon Throttles Netflix Subscribers in Test It Doesn't ... by jerkstate on Monday July 24, 2017 @01:12PM (#14840397)
let's do a little math..

the router ports on both sides, let's say you are using 4 ports across 4 fully loaded nexus 9508 (let's call those $500k per incl. optics, so $2M / 384 x 4 ports / 36 months = $600/port/month x 4 = $2400/4-ports/month)

Depending on how the connection between the two worked or was paid for, it could be $1000-$20000 for 2 pairs of fiber per month depending on distance. Let's take a middle of the road $10k average cost.

So for 10Gbps peak redundant capacity, you are at $12,400 per month. Netflix 4k stream is about 16 megabits, so you can fit 625 4k streams in $12,400 of cross-connect capacity. If you don't share costs, that's about $20 per user to support Netflix's business model that people seem to think it's Comcast's responsibility to pay. That's not even considering Comcast's last mile distribution cost or paying any salaries.

Streaming video services soak up a lot of network resources and putting all of the cost on ISPs is going to increase your ISP's cost and price. How many low-income families do you think would lose broadband if it went from $50/mo to $100/mo?

Re:Verizon Throttles Netflix Subscribers in Test It Doesn't ... by LukaAl on Monday July 24, 2017 @02:19PM (#14841179)
Hold on a second. Your reasoning has some issues:

1) The peering happens in Internet exchanges, there's no way a peering connection in an Internet Exchange costs 1000$, your range is simply ridiculous. Usually, it is fixed cost 1K or less depending on the locations of the router

2) Yes, you have to pay for bandwidth between the Exchange and your users. But guess what? That's what your users are paying for.

3) Not everyone stream movie in 4K at the same time. So you could optimize your bandwidth usage out of the exchanges. It is called "multiplexing".

Yes, the rest of the chain could be expensive, but that's what your clients are paying for. Net neutrality is about transparency. I have no problem in understanding that my 40$ connection will perform differently than an 80$ one. But you should compete in giving me the best service (where best is best for me, not universally) at the lowest price point in a way I could easily compare prices.

The problem, with my statement, is that implies Broadband Internet is a commoditized service. And that is all the battle about Net Neutrality, ISP doesn't want to act like commodities, they don't want to be your P&G or your ConEd. They don't have the structure to compete and instead of changing for the new market structure, they are fighting back. What they are doing is rent seeking. Good for them, bad for the economy...

Re:Verizon Throttles Netflix Subscribers in Test It Doesn't ... by jerkstate on Monday July 24, 2017 @02:49PM (#14841434)
I acknowledge that the costs are ballpark and not exact, but I was responding to a post that called the costs "negligible" - I was simply trying to quantify a slice of the cost and demonstrate that it can be a significant fraction of your bill. Of course your bill ALSO pays for the last mile infrastructure and connectivity to the hundreds of thousands of other routes carried on the internet. But it's clarifying to see that the data interchange component, which people are calling "negligible" actually is a significant portion of the cost.

My numbers have been called ridiculous by a couple of people now but they are based in personal experience, albeit a few years old, and nobody has posted any other cost breakdowns that demonstrate an understanding of the industry and the costs involved, just "those numbers seem really high!"

The difference between bandwidth and power is that data is NOT a commodity like power is, power is fungible and can be drawn and combined from a number of sources to fulfill the demand, but the dilemma with ISP bandwidth is that in order to satisfy customers the ISP must ensure adequate bandwidth to each individual content provider, and this is a much harder problem.

Re:Verizon Throttles Netflix Subscribers in Test It Doesn't ... by michaelmrose on Monday July 24, 2017 @03:13PM (#14841675)
The problem is your numbers are terribly wrong in multiple ways. I don't think people have mentioned that not every subscriber is online simultaneously or that most are still watching 1080p.
Re:Verizon Throttles Netflix Subscribers in Test It Doesn't ... by jerkstate on Monday July 24, 2017 @04:32PM (#14842394)
Your argument is at least as wrong as mine, though, while you seek to reduce the numbers, you are not also seeking to increase the scope of the cost from JUST the hardware depreciation and monthly fiber cost of the interconnect to the actual cost of the interconnect. The point is, it's not even close to "negligible"
Re:Verizon Throttles Netflix Subscribers in Test It Doesn't ... by FireBeyond on Monday July 24, 2017 @06:56PM (#14843499)
Not to mention if you're saturating a 10gbps connection, Netflix will throw as many OpenConnect appliances at you as you'd like. Yes, they're not "free" when you factor in racking, etc, but local bandwidth is cheaper than peering, even factoring in your valid comments here.
Re:Verizon Throttles Netflix Subscribers in Test It Doesn't ... by tw04 on Monday July 24, 2017 @02:18PM (#14841174)
Those numbers aren't even close to accurate. A fully loaded nexus 9508 (assuming the 10Gbe ports you listed) is ~300k LIST. I'd be shocked if a company that orders as much as Verizon does would pay even half of that.

As for the "distance" - they are sitting next to each other in a data-center. The distance is likely measured in tens of feet and the cost is likely a fixed cost of a couple hundred dollars. The optics on either end if they aren't close enough for twinax would add a couple more grand.

It's also ridiculous to use 36 months as your payoff date, they are running equipment a heck of a lot longer than that. The line cards and supervisor modules might get swapped out, but I'm willing to bet Verizon keeps their chassis level switches for 7-10 years on average.

So now you're at about 4$ per user to support Netflix traffic.

Re:Verizon Throttles Netflix Subscribers in Test It Doesn't ... by function_seven on Monday July 24, 2017 @03:43PM (#14841955)
Now throw in some standard oversubscription math, and you're at 40¢ per user. 625 simultaneous streams can probably serve roughly 6,250 customers, given that half the customers don't use Netflix, and the half that do, use it at different times of the day.
Re:Verizon Throttles Netflix Subscribers in Test It Doesn't ... by 4ad on Tuesday July 25, 2017 @10:34AM (#14848000)
Also the proportion of customers who needs 4k as opposed to 1080p is negligible. Most consumers don't even have equipment capable of displaying 4k content.

Netflix in 1080p uses 4.22Mbps or sometimes 7.15Mbps, so we're now at 15¢ per user.

Re:Verizon Throttles Netflix Subscribers in Test It Doesn't ... by mlrtime on Tuesday July 25, 2017 @08:23AM (#14847162)
Wouldn't they cross connect at 40 or 100GBe?
Re:Verizon Throttles Netflix Subscribers in Test It Doesn't ... by tw04 on Tuesday July 25, 2017 @08:51AM (#14847293)
They would most likely given the cost per gb would be significantly reduced, but I was just going with his stated scenario.
Re:Verizon Throttles Netflix Subscribers in Test It Doesn't ... by web007 on Monday July 24, 2017 @02:36PM (#14841339)
It's not Comcast's responsibility to support someone else's business model.

It's also not Netflix's responsibility to support Comcast's business model if they can't afford to provide bandwidth that customers desire and are allowed.

It's Comcast's responsibility to support their advertised usage sufficiently. If they say that they support 100mbps then I should be able to get 100mbps from wherever I like. They're going to have to pay for that uplink somewhere, whether it's Netflix or Google or Zayo or Level3 or whoever.

Re:Verizon Throttles Netflix Subscribers in Test It Doesn't ... by jerkstate on Monday July 24, 2017 @04:43PM (#14842489)
I agree, this is the crux of the argument. Broadband ISPs have been guilty of this for some time. Mobile players are generally getting a little more leeway by capping total transfer per month and not talking about rates as much. And T-mobile's "Binge On" product has been mostly hailed as pro-consumer, even though it does shape video down to SD bitrates and is a direct violation of the "paid prioritization" tenet of net neutrality as described in FCC's directive.
Re:Verizon Throttles Netflix Subscribers in Test It Doesn't ... by FireBeyond on Monday July 24, 2017 @06:58PM (#14843514)
> If they say that they support 100mbps then I should be able to get 100mbps from wherever I like.

Precisely. Not "from wherever we've got a 'preferred' agreement or kickback only".

Re:Verizon Throttles Netflix Subscribers in Test It Doesn't ... by shuntress on Monday July 24, 2017 @02:08PM (#14841055)
>that's about $20 per user to support Netflix's business model that people seem to think it's Comcast's responsibility to pay.

So what is my $80/mo Comcast Internet Service supposed to be buying me?

Re:Verizon Throttles Netflix Subscribers in Test It Doesn't ... by LanceH on Monday July 24, 2017 @03:28PM (#14841821)
Lobbying.

Seriously, though, could someone do a traceroute and tells me who pays for each hop?

Re:Verizon Throttles Netflix Subscribers in Test It Doesn't ... by jerkstate on Monday July 24, 2017 @04:55PM (#14842600)
Telecom and Internet lobbying aren't all that different: https://www.opensecrets.org/lobby/indusclient.php?id=B09&yea... https://www.opensecrets.org/lobby/indusclient.php?id=B13&yea...

Alphabet spends more on lobbying than Comcast.

Re:Verizon Throttles Netflix Subscribers in Test It Doesn't ... by wmf on Monday July 24, 2017 @04:01PM (#14842115)
Basically you (the customer) pay for the last mile and Netflix is getting extorted into paying for the rest.
Re:Verizon Throttles Netflix Subscribers in Test It Doesn't ... by Spivak on Monday July 24, 2017 @04:44PM (#14842501)
Not just the last mile but the peering agreements that your last-mile ISP has with either a backbone provider and/or other networks.
Re:Verizon Throttles Netflix Subscribers in Test It Doesn't ... by Karunamon on Monday July 24, 2017 @01:26PM (#14840574)
That's assuming that we're talking a connection streaming a video from Netflix's CDN to the customer's device. When in reality, Netflix offered to put a CDN cache box in Comcast's DC which would largely reduce the problem, Comcast turned them down.

That, alone, tells me that this is not a congestion or a cost problem, it is a bull-headedness for the purpose of rent seeking problem.

And as to the "last mile" costs, those are what Comcast subscribers are paying Comcast for. They act like this traffic is unsolicited noise, when in reality it's why they're being paid anything by anyone. I say don't charge people for x megabits down if you don't intend on them using it.

Re:Verizon Throttles Netflix Subscribers in Test It Doesn't ... by jerkstate on Monday July 24, 2017 @01:39PM (#14840714)
My understanding is that Comcast offered Netflix commercially reasonable terms for colo and cross-connect and Netflix opted for public interconnect via an exchange, which alleviated the bandwidth constraints caused by Cogent taking Netflix's money and then refusing to comply with their cost-sharing peering agreement with Comcast. Calling the Netflix CDN boxes "free" is totally misleading because Comcast would still need to pay for the router ports to carry the traffic internally, space & power, and management assistance. That deal works for a small ISP that pays for most of its bandwidth but not a large ISP that uses significant amounts of shared-cost peering. If Comcast is required to host those boxes "for free" what of every other CDN that they previously had agreements with?

I agree that Comcast marketing "up to X megabits" is misleading and those chickens have come home to roost with people thinking that their last mile means they should get that speed to every point on the Internet 24 hours a day. I have been negotiating commercial bandwidth agreements for years and even from a tier 1 you can't get that guarantee in a contract.

Re:Verizon Throttles Netflix Subscribers in Test It Doesn't ... by Karunamon on Monday July 24, 2017 @02:14PM (#14841120)
Calling the Netflix CDN boxes "free" is totally misleading because Comcast would still need to pay for the router ports to carry the traffic internally, space & power, and management assistance.

Which is still less than the cost of doing a proper interconnect - they chose instead to let it degrade and play semantic games instead. It also puts the lie to their complaint that it was ever about congestion. The correct response to people requesting a lot of traffic from X is to ensure that traffic is delivered efficiently. That is why their customers pay them.

Re:Verizon Throttles Netflix Subscribers in Test It Doesn't ... by bradleyjg on Monday July 24, 2017 @01:53PM (#14840882)
My ISP keeps on trying to get me to upgrade to "up to" 100 Mbps service. What in the world do I need that kind of bandwidth for except to stream videos? Shouldn't they disclose that if I upgrade I can't actually use the bandwidth unless the video provider pays them too?
Re:Verizon Throttles Netflix Subscribers in Test It Doesn't ... by Spivak on Monday July 24, 2017 @04:48PM (#14842551)
That does seem to be a trivial point. You're not going to get 100Mbps unless the entire path between the endpoints can push that speed.

You're getting a 100Mbps connection to the edge of their network and they have peering agreements, what else should they be doing?

Re:Verizon Throttles Netflix Subscribers in Test It Doesn't ... by FireBeyond on Monday July 24, 2017 @06:59PM (#14843522)
Eh, that's getting less and less of an issue. My nominal 105/20 internet connection from Comcast Business rarely pushes less than 80mbps download speeds, and from any major provider, CDN, regularly hits and sustains 120mbps.
Re:Verizon Throttles Netflix Subscribers in Test It Doesn't ... by Dylan16807 on Tuesday July 25, 2017 @04:45PM (#14851696)
> what else should they be doing?

They should be peering freely with anyone that is responding to requests from the ISP's end users.

Re:Verizon Throttles Netflix Subscribers in Test It Doesn't ... by mikeash on Monday July 24, 2017 @01:40PM (#14840723)
That's not $20/user to support Netflix's business model, that's $20/user to support Comcast's customers' usage model.

If you think this is unfair, then the answer is to get rid of bullshit "unlimited [but not really]" connections and charge people based on what they use. Stop making low-impact users subsidize people who stream HD movies 24/7.

Re:Verizon Throttles Netflix Subscribers in Test It Doesn't ... by FireBeyond on Monday July 24, 2017 @06:55PM (#14843490)
> that people seem to think it's Comcast's responsibility to pay

Funny, the upwards of $50 to $200 I pay Comcast per month should go somewhat to that.

Besides, how many users are streaming 4K content, on what little 4K content Netflix has (I'm going to wager that we're looking at about 5% or less).

At 5mbps for 1080p, peak, now we're at 2,000 users, and $6/month.

Is it really that onerous for most users of Comcast to expect that $6 of their say $60/mo cable bill goes to Netflix?

I'm also not sure why you're populating 4 ports and then only talking about 1 10Gbps connection, when in reality (though I'm no expert), that's probably 4 10Gbps connections. Accounting for the cost of 4 populated running ports and then only talking about the capacity of 1 of those ports when calculating cost/megabit seems misleading.

Re:Verizon Throttles Netflix Subscribers in Test It Doesn't ... by michaelmrose on Monday July 24, 2017 @01:39PM (#14840716)
Back in reality the huge ISP's provide network services at a huge premium and make a very substantial profit. If we cared about providing affordable service to low income families we would let municipalities sell access to existing fiber networks not trying to prop up an industry providing bad service at monopoly prices.

Your numbers are like historical fiction based on reality but not an accurate portrayal. Do you work for verizon or comcast?

Re:Verizon Throttles Netflix Subscribers in Test It Doesn't ... by jerkstate on Monday July 24, 2017 @01:40PM (#14840729)
Right, and Netflix and Youtube are charities, give me a break with that argument.

since you INSISTED, I have no personal interest in any ISP

If you're going to refute my numbers please provide your own.

Re:Verizon Throttles Netflix Subscribers in Test It Doesn't ... by michaelmrose on Monday July 24, 2017 @02:51PM (#14841452)
Source your numbers. Most specifically that netflix which can provide the entirety of its service for 10 per subscriber actually costs providers $20 per subscriber.
Re:Verizon Throttles Netflix Subscribers in Test It Doesn't ... by mrkrabo on Monday July 24, 2017 @12:44PM (#14839993)
That's what splitting the bill means.

You don't pay "for the bandwidth", you pay for a link between two ISPs (Google and Verizon). The bill between then should be split, otherwise the traffic will have to pass through somewhere else, and that will cause congestion problems.

Re:Verizon Throttles Netflix Subscribers in Test It Doesn't ... by jmuguy on Monday July 24, 2017 @01:08PM (#14840347)
Google has most certainly paid for bandwidth/traffic from their end. Whether that's to a CDN or some other arrangement, and there's likely already peering agreements in place. In these cases what Verizon is doing is asking for money above and beyond what it actually costs to handle the interconnection. They use declining service to their own customers as leverage. Its ridiculous, and even more worrying when you see ISPs and content providers merging as is the case with VZ buying Yahoo and AOL, Comcast and NBC merger, etc.
Re:Verizon Throttles Netflix Subscribers in Test It Doesn't ... by michaelmrose on Monday July 24, 2017 @01:33PM (#14840653)
Google pays for their own network services. Whomever google pays for network services pays pays if applicable to connect to Verizon's network.

The sole and only thing they are being paid for on the ISP side of Verizon's operation is to quickly and reliably deliver the content people want to consume over the pipe people have paid to have installed.

YouTube by hosting content people want to consume is driving demand for Verizon's services.

Pretending that this is a cost is a bazaar inversion of reality.

Re:Verizon Throttles Netflix Subscribers in Test It Doesn't ... by criddell on Monday July 24, 2017 @03:06PM (#14841591)
> Google pays for their own network services.

Do they? Who do they pay? I thought that maybe one of the reasons Google got into the ISP business was to be a peer with the other big networks and not pay any peerage fees.

Does Verizon pay for network services?

Re:Verizon Throttles Netflix Subscribers in Test It Doesn't ... by jessaustin on Monday July 24, 2017 @04:38PM (#14842439)

  s/bazaar/bizarre/
Re:Verizon Throttles Netflix Subscribers in Test It Doesn't ... by mtgx on Monday July 24, 2017 @02:55PM (#14841501)
Only Google and Verizon? Top 10 content providers? Top 50? top 100? Who would set the rankings? Should everyone split the bill with Verizon?
Re:Verizon Throttles Netflix Subscribers in Test It Doesn't ... by mrighele on Monday July 24, 2017 @01:13PM (#14840414)
Because Verizon's users are already footing the whole bill. If anything, the bill should be split between the end user and Google.