Costs for consumers are whatever the ISPs decide they are. The answer should look like "if you cost us more, we charge you more," just like every other business on the planet.
It doesn't have to be X bits = Y dollars. It could factor in load; the meter runs faster at high-demand times and slower at off-peak times. That solves the "bandwidth is a renewable resource" argument and the "some people use more than their share" argument. They'd just have to ensure that the meter is highly visible to consumers.
Contrast this with Datacenter bandwidth where paying $.10 a megabit isn't unheard of. The cost structures are not related to the core infrastructure, but rather the edge.
I think this problem is a lot more complex than you're making it out to be, but that's just my opinion. In a lot of cases the rates for ISPs aren't set randomly, they're anchored to standards set by regulatory bodies (particularly in the case of DSL).
In short, pricing bandwidth is hard. The real cost is the last mile.
If so, that doesn't make me sympathize with them at all. If they want freedom to charge more according to access, or to charge more according to the difficulty of running the last mile connection, I'd say "give up your monopoly protections and we'll remove your regulation."
If they don't want that, I'd say "you start charging both the consumer and the content provider and we'll pass a regulation saying you can't."
If a company wants the government to be their sugar daddy, they shouldn't complain when they get grounded. You want your own rules, you move out of the house and compete in the free market.
At the consumer level, competition is scarce. In the SMB Segment competition is fierce. In the Datacenters world competition is fierce, but at the core of everything you have to go through 721 Folsom street to get out of the left coast.
This fight is particularly interesting because it's one of the first to cross the Commercial-Residential boundary. On the one hand, consumers pay for access, on the other, large ISPs peer networks. One is a contract, the other a handshake.
In this particular case, ISPs argue that since usage increases so should rates, but in order to arrive at such a conclusion, the large ISPs would have to ignore their monopoly.
Let's take this a step further though; there's a really big problem with everyone building their own network. That is, no one wants thousands of wires running to each residence, but that's what a competitive last mile would look like.
The answer is municipally controlled networks with private operators. The infrastructure would be owned by the state, but operated by competitive virtual operators.
Unfortunately, this vision is impossible in America, and that's why we can't tackle this problem straight-on and instead have to operate through layers of abstraction.
TL;DR: public networks, private operators is ideal. Impossible in US.
IMHO the incumbent carriers have no interest in competing if they can be a monopoly.
But I see it differently the relevant scarcity is over bandwidth: total data transfer at any moment in time. That is, downloading when everyone else is "hurts" the network a lot more than when you download when it's below capacity.
So why not just throttle heavy users when at full capacity, and only then?
I know a lot of people here don't like anything involving throttling, but it only seems fair that (if you don't want metered usage), then people who put less load on the network should get priority when there's only a scarce amount to go around. (It should be weighted, of course, by how much of your usage was on or close to peak hours so that you're punished less if you schedule your torrenting for off-peak.)
What am I missing?
Access is scarce. This is not a scarcity problem, this is a distribution problem. Bandwidth inside of a Datacenter can be had for $.10 per mbit, but because access to your home is prohibitively expensive, you pay more.
Does that help? The key point is that it's a distribution problem, not scarcity.
So what is it that the ISP cannot provide, specifically? I'm guessing it's the sum of all data passing to/from its customers per unit time. Hence why I called bandwidth the relevant scarcity, and suggested apportioning it based on how much users are using the network.
Can you operationalize the term "distribution problem" so that it's clear what specifically the problem is? "It's a distribution problem" is easy to fall back on, but hard to make precise. They say the same of food shortages, for example, but people have different ideas of what it means.
I think the problem is that the last mile access is prohibitively expensive. Glass connections in a datacenter are $.10 per megabit, but AT&T DSL over copper to your house is $30 for 1.5 megabits. That copper could become glass, but no one will pay for that upgrade.
The distribution problem is that in order to compete with at&t at the residential level, you have to build a parallel connection and the cost of doing so is prohibitively large. Therefore, everyone is forced to rent access from AT&T, BUT since common carriage doesn't apply to residential access lines, AT&T doesn't have to share.
If sonic.net could build glass to your house, they would, and they wouldn't ever complain of congestion, but because of artificial barriers to entry no new competitors can enter the market.
Does that make sense? I'm not trying to be evasive, it's just a complicated problem. Carriers don't have a problem dealing with aggregate capacity, they have a problem dealing with edge access capacity. Core carrier routers like the CRS-3 from Cisco can move 322tbps+, and they do, but you can't put a CRS-3 in every DSLAM.
Forget for a moment about technology/capital upgrades, and assume that the ISP wants to give everyone as much speed as possible. The customers collectively try to download as much as they can, as fast as they can. Naturally, they'll hit a limit. At what point does the limit affect their usage, and how can the ISP fiddle with access so as to discourage overuse? And why is my solution not one possible method (if suboptimal)?
Let me try again.
When we talk about wireless bandwidth, there is a big problem with congestion. When we talk about residential bandwidth, congestion isn't the issue so much as competitive access is. It's not that AT&T and Comcast have the best access, they just got there first and are able to block other competitors.
Throttling will make the ISPs more money, but it doesn't help consumers or solve this problem.
I think this might be explicitly what you're looking for: At the pricing and access tiers available today it is virtually impossible to saturate and block a local connection (at the residential level) such that you impact other subscribers at a regional level.
Throttling doesn't solve this problem it just reduces usage. ISPs have the overall goal of reducing usage because it increases profit. The logical fallacy is that while your model proposes an ISP that would want to give everyone as much as speed as possible, the reality is that all ISPs want users to use as little bandwidth as possible.
Does that help? Really I'm not trying to show off my vocabulary it's just a complex subject :(.
In AU/NZ at least (and I am simplifying) you have the last mile, the trunks between exchanges, the handover points between big exchanges and ISP POPs, and finally you have the ISP upstreams - any of which might be a bottleneck if sufficient traffic is generated.
In AU/NZ (and much of the rest of the world) you have local peering exchanges etc, but ISPs also have this cost for "upstream", which is money they have to pay to their friendly international cable monopoly to get bits from faraway places.
These backhaul links are not normally configured to handle all users running at full rate. That would be expensive, and silly.
In the fixed line world, you might observe "congestion", but it the root cause is usually "oversubscription".
Everyone oversubscribes backhaul and upstream, the question is how much you can get away with. The reward for guessing lower than your competitors is lower costs (or being able to defer an upgrade cycle). The penalty for guessing way too low is churn.
I say "guess" because even though telcos tend to have good models of user behaviour, user behaviour can change very quickly (from a telco's POV) when new services come on line.
Such as online video...
I kinda expect unlimited data plans to be a very short lived phenomena with the possible exception that it may end up free where the traffic can be offloaded to a wifi.
It's important to note just how much video content has grown compared to revenues though. If you click through to the link on the Deck, you'll see a bunch of stats about video utilization compared to revenue. The numbers are pretty mindblowing.
Costs for consumers haven't really gone up, but utilization has increased considerably. Any idea about what the answer should look like? (Note: you can't say Google Fiber).