If peering is untenable because it leads to outcomes like the one we have now (Netflix) where it is fundamentally unfair to one party (assuming that is the case) or the other, then we're all screwed. There might not be a fair solution that still manages to resemble the Internet.
In one alternative, the Comcasts and the AT&Ts are smited and no longer exist, and Level 3 goes into the consumer internet business, connecting things end-to-end. That's not something we could trust them with. Same if Comcast replaces Level 3.
In another, Netflix continues to pay the extortion (if it is that) which we have little doubt that Comcast will continue to ratchet up the price for. Or Comcast has to shoulder that burden alone (and can't charge customers extra for doing so, without people screaming "net neutrality!").
Or maybe services like Netflix just can't exist in such an environment. Also a bad outcome.
Or god help us, internet infrastructure is nationalized, and the same people who manage our roads and traffic lights take over.
Am I blowing this out of proportion?
There are many ways to skin an onion, and full-scale nationalisation is probably the least likely outcome (cash-strapped governments don't need another headache right now). In fact, the real problem here is that one player is leveraging a monopolistic position; remove that position, and the Free Market should start working its magic again.
Serious question, how would one find the exact sum with which Comcast should be compensated in the absence of a functional market?
Thankfully, that company and environment are gone. Let's not recreate them.
Are you talking about on this graph?
http://blog.level3.com/wp-content/uploads/2014/05/route_info...
Because all the numbers I see are showing about a 5:1 or 6:1 imbalance.
> ...transit providers are starting to see their industry be squeezed by the big ISPs.
No.
> ...why do transit providers even exist?
Because it's untenable (and inefficient) for Comcast to build a separate fiber network to every service provider (e.g. to start providing a service on your version of the internet, you'd have to build your own network connection to Comcast, and then again to Verizon, and again to AT&T, and again to every other ISP on the planet).
Take another look at the L3 network map from the blog post:
http://blog.level3.com/wp-content/uploads/2014/05/network_ma...
If you think transit providers are purely middlemen for the sake of middlemen then you're basically saying that every service provider should build out a network of that same size in order to reach every ISP.
> Settlement-free peering assumes that bandwidth usage is roughly symmetric
No. It assumes that the benefit to each network is roughly symmetric. Put another way, if Netflix videos only degrade on Comcast's network, Comcast's goodwill with its own customers will increase as a result of increasing transit/peering capacity. To a normal company, that's a tangible benefit to the agreement enough to make it worthwhile.
Comcast empirically does not care about the goodwill of its customers, which is not surprising given its monopoly of local markets. If customers had alternatives to Comcast and it became known that Netflix worked on one ISP and not the other, customers would switch ISPs en masse.
I also used a bit of hyperbole in the "why do transit providers even exist" part; transit providers do still need to exist to provide access for the long tail. But Level3 and the like do see their market shrinking as the big fish who currently pay them for transit either directly or indirectly move more bandwidth to direct connections with the big ISPs. It is definitely starting to squeeze them.
As far as the benefit to each network goes, that's hogwash. It's not how the contracts are written because you can't quantify "benefit". You can quantify packets sent/received. Netflix needs to make money just as badly as Comcast does, so the fact that their service is crappy is much more their problem than it is Comcast's. Netflix can always purchase transit through another company that's not Comcast.
Historically, eyeball networks have paid for transit from backbone providers. As market consolidation occurred, the cable providers with government granted physical monopolies were able to negotiate for better settlements with their transit providers.
Eventually, the cable companies condensed with the major telephone companies. Some of the new mega companies have backbones and can use their own networks for transit (Verizon, AT&T, ...) others (Comcast) were able to use their size and access to their customers to negotiate largely settlement free exchanges.
However it is important to note that ALL eyeball networks have 1:5 to 1:20 demand ratios. This is the nature of content versus consumption. There is no new news regarding these ratios, and they are not particularly germane unless attempting to engineer the flows.
The real and pertinent issue is that Comcast has not lived up to the 'timely upgrade' clauses in boilerplate (NDA'd) settlement-free peering agreements. Clearly they have an advantage to "defect" from the standard cooperation model. Netflix has chosen to change providers several times, and recently provided data on whom they pay for service.
...you can't quantify "benefit"....
Isn't that the entire purpose of an economy?
Can you qualify this a bit? What's the actual expect cost to Comcast here to fix the issue? General ballpark?
But there are also typically shared costs for networks to
interconnect. Each party pays to augment its own network
to allow for more traffic exchange (the expense to augment
capacity is not significant for either party). And since
we often choose to interconnect in a third party data
center, the networks usually agree to share the cost of
the cross connects by paying for them on an alternating
basis.
I take that to mean the CapEx is insignificant and the recurring expenses are shared equitably. What's notable about that is that Comcast isn't even sitting down at the table to negotiate equitable terms for upgraded capacity. They're out and out refusing to upgrade without direct payment.I've never understood why this is - especially in a case like this, where the sending side is sending because (the customers of) the receiving side explicitly asked for it.
The assumption is that if you're sending traffic to a customer, you are being paid for that activity through subscription fees, ads, donations, etc. That packet costs the receiver some minuscule amount to process. Even though it was requested by the ultimate receiver, the ultimate sender is only sending the packet because they make money from doing so. The middlemen only pass on the packet because their costs are covered and they make a small amount of profit. This setup also ensures that if the packets aren't being monetized in some way (i.e. if it's a DDoS) that there's a financial incentive to stop them.
Last time I checked ISPs where heavily monetizing packet receiving to the tune of 5-10x what netflix monetizes from me. For typically slow, shitty service bundled with cable TV or phone I do not want.
That's true for hot-potato routing, but CDNs generally use cold-potato routing.
As for monetization, both Netflix and Comcast are getting paid by their respective customers.
What it comes down to is Comcast has no incentive to ensure its routes to various Internet transit providers are GOOD if they're not monetizing them. Why should Comcast be treated differently than any other transit provider and not allowed to monetize its transit services? Netflix wasn't paying Comcast, so Comcast had no responsibility to help Netflix make its own service better when there were other ways Netflix could have alleviated the bandwidth situation (e.g. buy transit through another company).
Customers pay Comcast with money to establish those links. If that's not "monetizing", what on earth is?
Comcast will get paid by their customers as long as the routes they pass traffic through are 'good enough'. They have a (near) monopoly on the last mile in many markets, so 'good enough' can mean both 'barely working' and 'better than any other option you've got' at the same time.
So, you're right, but so is the parent: Comcast has no financial incentive to provide any more than just enough bandwidth to keep you from calling the support line to complain or cancelling your contract in frustration.
Where's the rest of customer revenue going, if not towards network upgrades? Well, let's just say the Comcast Center (http://archrecord.construction.com/projects/bts/archives/off...) ain't paying for itself...
Edited to add: It's certainly true they are not rewarded for the marginal packet, but then neither is Netflix (given their subscription based model).
Does it assume that?
Since about the time of the web browser, we've had consumer-focused ISPs. Consumer traffic is mostly small requests for large replies. But as the article suggests, Level3 has a lot of settlement-free peering with consumer ISPs. My belief is that being settlement free isn't about equality of packets, but equality of demand. E.g., I have a lot of servers made to serve consumers; you have a lot of consumers wanting access to my servers; let's just split the costs.
In any case, given that those Comcast packets are mostly requesting all those Level3 packets, it seems much fairer to charge Comcast disproportionately, as its their customers who are creating the demand.
Even if you want to privileged "sending" as being more charging worthy since you control what you send— in this case it's Comcast customers that have requested those bits, hosts connected to the level3 network are providing them.
Networks like comcast have built out imbalanced networks which likely wouldn't exist without their access to monopoly infrastructure. Practically anyone they peer with except other consumer broadband monopolists is going to be unbalanced.
Due to using many distinct flows it also tends to not be very TCP friendly and takes a fairly unequal share of the bandwidth. And while users will give up and do something else when their interactive service becomes slow, a congested torrenting host keeps torrenting.
What's happening here is that Level3 and other transit providers are starting to see their industry be squeezed by the big ISPs. If you have 5 ISPs that serve 90% of the customers in the US, and 10 service providers that generate 90% of the bandwidth, why do transit providers even exist? I think they see their market share decreasing significantly as more of their customers follow in Netflix's footsteps, so they're trying to pile on and start a grass-roots outrage like Netflix did. Problem there is that most people haven't heard of Level3, so it makes it more difficult to get people behind them.
And you are completely right about the whole Netflix issue being Netflix and Level3/Cogent's problem. Level3 and Cogent have business models based on selling transit acquired through settlement-free peering. Settlement-free peering assumes that bandwidth usage is roughly symmetric: when it's not symmetric, the side sending more bandwidth has to pay. Level3's position here seems a bit hypocritical: if they had a customer that was routinely sending data at a 20:1 ratio, they would charge that customer for sending more bandwidth. But they're expecting the big ISPs not to do the same to them?