And then there's this: http://arstechnica.com/information-technology/2014/04/bittor...
"[W]hen we ask [ISPs] if we too would qualify for no-fee interconnect if we changed our service to upload as much data as we download—thus filling their upstream networks and nearly doubling our total traffic—there is an uncomfortable silence," Netflix CEO Reed Hastings wrote last month. "That's because the ISP argument isn't sensible. Big ISPs aren't paying money to services like online backup that generate more upstream than downstream traffic. Data direction, in other words, has nothing to do with costs."
That's because around the time Comcast was building up their networks customer demand was wildly asymmetrical. People simply download far more than they upload and Comcast wisely dedicated more of it's bandwidth to downloads.
At the time all cable companies were were building up their data network, they were building that network over top of their existing video distribution network. That network was designed [ages prior to this point] to have tiny, tiny, upstream channels for use by system control and maintenance devices, and huge downstream channels to shove 100+ analog TV channels. Because it would have cost a metric shitload to replace all of that hardware, the cable companies were forced to give their data subscribers pretty large downstream links and relatively meager upstream links.[0]
Frankly, the nerds of the day (remember, this is the mid-to-late 1990's) would all have been running home servers, and would have been demanding a symmetric connection so that they would have a substantially cheaper alternative to ISDN.
(Note that nerds that remember those days consider asymmetric home Internet connections to have been more harmful than the Eternal September.)
[0] Note that this property is pretty meaningless in an HFC network where the cable company could run fiber directly to the home, but chooses not to. [See also ATT U-Verse.]
I guess the core issue here is that, from a customer's point of view, the ISP is selling me inet access, regardless of where the data actually goes, so it is the ISP's responsibility to keep their end of the bargain (to provide inet access). As a customer I should not care how is their topology planned, what kind of equipment they use, etc.
So basically the ISP though that they could offer T amount of bandwidth, advertised it as U (with U > T) because most customer won't use even close to U, and now that more and more users are actually using U amount of bandwidth (or closer to it) the ISP's infrastructure is suffering because they simply where not prepared for it, i.e. they oversold their capacity.
OT: By the way, why downvote the parent? I really thought downvoting is for when the post is off-topic or offensive or something... but more and more I see HN modders downvote because they disagree... really? so you make the post go into an ureadable color (grey) just because you disagree? I don't think it's supposed to be that way, is it?
An airline for example; now everyone actually caches their flight because Google's self-driving cars always make it on time, and then they demand that Google stops bringing passengers on time?
Or a buffet restaurant, since now everyone goes to the gym, they can actually eat a lot more, thus restaurants complain about, either the gym or the pill maker?
I don't know, it would be a strange world to live in if this happened in other industries. Maybe prices would rise too much?
Fast forward 3 years later, you come out with your pill and throw a wrench in the works. Now gyms realize they have to go back to the old pricing model because the economics have changed. Consumers will no doubt become upset at the prospec of having to pay more. Some gyms take a wait and see approach to see what other gyms will do. Your pill started it but really, it's not your fault or your problem.
What should the gyms do?
Be Honest. They can invent a new business model based on transparency. Imagine if price is not fixed but you are charged per use based on the traffic at the moment. It could be similar to what Uber does, price based on demand.
In terms of ISPs, I think there should be a micro transaction exchange where I can buy X minutes of data transfer. If Verizon is clogged I switch my router to connect with ATT etc. All networks should create a single broker device thats given to each consumer.
Funny, I actually just added an edit to my post above that discusses this. Comcast adds interconnect capacity as needed. So in at least one way, it's actually easier this way, right? Just one peer to make sure they have capacity for.
But either way, it's Comcast's job to run a network that's optimized for its customers needs, not some abstract concept of fairness. And in this hypothetical, it's not on Netflix that streaming lots of content is not compatible with how Comcast happened to build its network. Comcast needs to fix its network so that its customers can get their data.
But Netflix is not "just a small number of nodes". They're basically their own CDN; they have servers all over the Internet, precisely in order to not have the problem you describe.
What's more, Comcast has a deal with Netflix to directly connect Netflix nodes to Comcast's network; in other words, Comcast directly controls how many paths there are for Netflix content to get into its network. If there aren't enough such paths, they can just open up more.
You can even ask them for a freeish box to run inside your network to reduce requests to their CDNs
Consumer ISPs have never expected symmetric bandwidth. If they had, they would sell us 50/50 connections instead of 50/10 or 50/2.
The Tier 1 always wants to ask for more bits than a lower tier operator. If a lower tier operator asks for more bits and tilts the traffic balance, things get interesting. Cogent is not a tier 1 but slings 35% of the internet around at will.
It does cost a lot of money to route Netflix's bits and the narrative is hugely helpful to Netflix. I'm not saying this is correct, I'm simply trying to point out that Netflix has a ton to gain from carriers upgrading their network connections and routers.
Another way to view this: Netflix is demanding huge network upgrades without any additional revenue source.
I think a lot of us would have more sympathy with the ISPs' situation if they hadn't consistently and for a very long time been promoting their services as ever more valuable by offering ever higher bandwidth, yet keeping quiet about little things like massive over-subscription and high contention ratios that meant if all their users actually tried to use the service that was purportedly being offered then no-one would get anywhere near the advertised speeds they were paying for.
With the rise of things like streaming video services and Internet telephony, some people are actually trying to pull significant volumes of data down the pipes relative to the advertised size of pipe available. Everyone's known this was coming for a long time, and no-one more so than the ISPs. The way I see it, if the current situation hurts ISPs that failed to invest or set a realistic pricing model, they'll just have to take their medicine and offer a more realistic deal to customers next time (or fail because they can't compete effectively).
Sounds to me like just another way of rephrasing "ISP Customers are demanding huge network upgrades without any additional revenue source."
Netflix does not send packets to you unless you request them.
Edit: not sure why I'm getting downvotes since silverstorm clarified exactly what I wrote
I am not talking about imbalance in U/D, but rather imbalance in destination (or source). If tremendous amounts of traffic is going to or from just a small handful of nodes, you need tremendous routers & pipes going to those nodes.
You're missing a key element of the situation: Comcast made a deal with Netflix precisely to address this problem. Netflix now has direct connections to Comcast's network, so Netflix traffic does not have to go through a third party. In other words, Comcast now directly controls how they connect to Netflix, how many connection points there are, what the average network distance is between the closest Netflix connection point and their customers, etc. So they have a much cheaper way to even out Netflix traffic: open up more connection points.
(Which, btw, AFAIK they already did as part of the deal; they certainly did not come out of the deal having "just a few spots" where they are connected to Netflix. Netflix basically is their own CDN; they have servers with their content on them all over the Internet. So Comcast has all kinds of ways to shorten the average network distance between each of their customers and the closest Netflix endpoint.)
Then Netflix comes along, and suddenly 35% of Comcast's customer traffic is all going to a handful of Netflix DSNs? Suddenly Comcast has to redo the whole connection. They have to buy top-shelf hardware just for Netflix.
I don't know if this has happened, but it's an interesting thought experiment to see how "just one company" could place disproportionate burden/cost on the ISP, as compared to the case where traffic travels to & from many different entities.