Thanks for the points. A few further thoughts.
1. There are two sides to every connection. Couldn't Netflix itself traffic shape?
2. Can I pay for QoS from an ISP for elevated service (for instance say I also have VOIP service for my office and want low-latency to trunk)? Doesn't my elevated status neccessitate shaping of others packets (for mine to arrive with lowest latency, others must be delayed)?
3. Lastly, given that all communications (everything from postal mail, to TV, cellphone, etc) relies on oversubscription, QoS traffic shaping is baked into the pie, there is no network access without shaping. Literally every single packet is shaped constantly along the way. Given this fact, QoS is the not optional. It is in fact the entire business process. Thereby, how could an ISP ever be shown to not be engaging in "reasonable network management"?
NOTE: I'm asking these questions somewhat rhetorically. My point always when NN discussions arise is to try to engage in actual technical discussion about the matter for the purposes of illustrating to others that its largely an undefined, impossible to enforce concept. In fact, the idea is impossible. To my repeated disappointment, few people - even on technical places like HN - bother to think about the technical aspects. To me this shows that NN is like a mind virus. Its the 2015 version of "won't someone think of the children" from the 80's. It is a crafty bit of wordsmith that makes it impossible for people not to support. Who could be for a "non-neutral" net? All of this is crafted for the politicians benefit, since they gain the power (and their monopoly lobbyist).
It's not like Netflix is generating petabytes of unsolicited traffic. They're providing the traffic Comcast's (or verizon's, ATT's, or whose-ever) customers are requesting. Net Neutrality is Wheaton's Law, writ large:
Don't be a dick. (To your customers, or the content-providers who produce what your customers want.)
Westinghouse (e.g.) can't charge me more for power to run GE (e.g.) appliances. Comcast (e.g.) shouldn't be able to extort customers for access to (e.g.) Netflix.
But if the user can get a faster speed through a VPN than without (as it has happened with Netflix vs Verizon), the tubes are NOT the limit, but a clear violation of Net Neutrality.
[0] As in: if shaping is done, you don't look at the source, destination or content.
Each network provider that the packet is passed thru inspects the destination address in order to determine which pipe to send it on to get it closer to its destination. If that pipe is congested, the packet can't turn around and find another route. It has to be dropped and retransmitted. This is true for streaming video packets as well as others like email, video game, etc (although most video games use UDP which cannot retransmit, if it's dropped, the game jitters)
So if I'm an ISP downstream from a congested pipe that happens to be transiting a lot of streaming video, I know that all of the users of that pipe who are not streaming video are having a bad experience - slow text web page loads, jittery video games, garbled voip calls, etc. So what I can do is selectively drop some of packets from the video streams (I can tell which ones the heavy users are if I record the source IP and count the # of packets per stream), and those streams will generally gracefully downgrade to the next lower streaming quality. This relieves the inbound congestion because the sender stops sending as much data to users downstream of me, and allows my other customers to play video games without lag, make voip calls, load web sites without long delays, etc.
So clearly the pipe needs to be upgraded, the question is, who should pay to have that pipe upgraded? That's between the ISP and the transit partner, they have peering agreements to determine how that cost is shared. I think it's important to look into who failed to live up to their contractual obligation to shoulder the shared costs.
BTW - going thru a VPN on Verizon just means that the traffic is getting to the customer through a different upstream provider than the one that BGP specifies because it originates from a different network, one that isn't as congested as the BGP route from Netflix. You would need to look at a traceroute to figure it out.
Also:
> [Level3] tells me that it actually offered to give the necessary hardware to Comcast (at around $50,000 per port) and that it offered to do "cold-potato" routing deep into Comcast's network, dropping off streaming traffic near the customers who requested it, for instance.
source: https://arstechnica.com/tech-policy/2010/12/comcastlevel3
> Who paid for the delivery of all this on-net traffic, then? The customers. In Level 3's case, this means that CDN customers like Netflix would pay Level 3, while Comcast's cable modem subscribers would pay Comcast. Very simple, very clean, and according to Level 3 now, this is the way the Internet should be connected.
> But after winning the Netflix deal this autumn, Level 3 suddenly wanted to pass far more traffic over its links with Comcast. Comcast balked; Level 3 suddenly looked less like a transit vendor and more like a CDN.
You are correct that they had an existing settlement-free peering agreement. But that agreement had traffic ratio requirements which Level3 broke because of its Netflix contract. So they can offer whatever else in the world that they want to seem like "nice guys" - free routers or cold potato routing or whatever. But Comcast is not obligated to allow Level3 to break the terms of their agreement so Level3 can make tons of money off Netflix at Comcast's expense.
- At least according to Level3, it was not standard practice in the industry to charge for peering (not transit - the final destination for the packets was in Comcast's network), regardless of traffic ratios.
- You could say that it should be standard practice, but that would be unfortunate because Comcast effectively has a monopoly on fast connections to its own customers. Since it's much easier for consumers to switch video services than ISPs, failing to reach a deal would be much worse for Netflix/Level3 than for Comcast - so there would be little preventing Comcast from charging as much as it wanted.
This was more true in 2010 when Netflix hadn't yet started creating original series. These days it arguably has more leverage due to being the exclusive provider of those series... but it would still be a huge risk, a huge money-loser for them to walk away from Comcast, or any other ISP, and get into a cable-blackout-like situation. Anyway, cable blackouts feel distinctly unlike how the 'open' Internet is supposed to work. Netflix is one thing, but what about, say, would-be competitors to Netflix that currently have few customers and almost no leverage?
- It would also arguably be double charging - charging consumers and backbone providers for the same packets. If you buy "Internet access" with 100Mbps download speed, but that actually means "100Mbps down from companies that pay the toll", are you really getting what you paid for? Admittedly, transit providers have similar practices, but the level of competition for transit makes that less of an issue.
- Also, most Comcast end-user connections have much higher download speed than upload speed, so even if users were saturating their (paid-for) connections in both directions talking to Level3, Level3 would still be sending more data than it received. In other words, rules about sending and receiving similar amounts of data never really made sense for peering between a consumer ISP and a backbone provider, even if this was less visible in the past.
I agree it sucks to be a consumer and get shafted by these shitty deals, but let's at least be honest about how we got here and what the powers that be are actually gunning for.
I'm genuinely curious: What do you believe the powers that be are gunning for?
NN means an ISP cannot place artificial speed limits on packets that I send or receive, based on the contents of the packets; e.g. the protocol, the port, or the destination address.
NN rules have already existed for several years, for the full specification see http://transition.fcc.gov/Daily_Releases/Daily_Business/2015... . This is a widely accepted concept so if you want to support your argument that net neutrality is "impossible" you'd need to rebut what this actually says.
To be clear, I don't disagree with your post. I just chose it as an outlet to vent my frustration that language and words are so hard.
At the very least, instead of saying "see, I told you it wasn't easy," why don't you summarize the responses you have received and formulate a set of rules that would satisfy as many as possible?
The percentage of content providers that deliver to telcos directly is almost zero. Netflix is one of the biggest notable ones, but even they don't always do it.
That said - a peering agreement can have a variety of terms so a gamer isn't likely to be privvy to that. But if you see latency and loss on a certain link, you've likely identified the problem. Typically both parties are bound to solve it if its the actual peering link, a fair amount of the time its an individual ISPs problem so that ISP would be obligated to upgrade its internal link(s) to meet the demands of its customers.
Some time ago (2014) I convinced (forced under contract) a very large last-mile carrier to fix/upgrade its internal links after I browned out a few counties with my paid-for usage. Its not that hard, it just took a lot of phonecalls on my part and calls with our finance department... I guess the alternative was for them to let me continue browning out their customers for a section of the San Francisco bay area, having them ragequit and switching to someone else. Or wasting thousands and thousands of hours with customers complaining about terrible service.
Good on you for getting a big telco to live up to their agreement. I doubt that lack of tenacity is the problem in cases having to do with large isp/video streaming/cdn players.
So Comcast shaping Netflix when Cogent won't pay their share to upgrade their peering point is probably still legal, but they would probably have to go to court to prove that it was "reasonable network management" and depending on the technical prowess of the court, could easily go the other way. The outcome is that telcos lose a lot of negotiating power to content providers.
More last mile providers probably wouldn't solve this problem, because people pay money for what they actually want (streaming video) rather than what they say they want ("free speech") and an ISP that protected freedom of speech would be protecting nazis, white supremacists, fake news, etc.