If you use Boost ISP and watching sports on NBCSport it will work fine but watching the same match later on youtube is not. If you sit at CSpire and watch a paid movie on Youtube Red you will get worse experience then if you paid and watched the exact same movie on Netflix. If you sit at GCI and have both Netflix and Amazon Prime then go and unsubscribe your netflix as it will be much worse experience than amazon thanks to your ISP throttling.
This is not small ISP that monitor overall usage and throttle clients based on high usage and hotspots of traffic. The article is not about congestion control but rather marketing choices where one streaming service is favored over the other within the captured market of an ISP customers.
They'll clearly tell you that faster is better:
> Indulge in super speed and have a great online experience, whether you're browsing the Web, shopping, or streaming your favorite movies and shows.
But I want to know what my money is actually paying for.
You pay your local delivery company $X per month and they promise that they'll deliver you 1 package a day from their local depot.
But it turns out that although you're ordering a package a day from Amazon, you're not receiving a package a day. You get your stuff, but it's delayed and backed-up somewhere.
And so as an experiment you sign up for a service that will receive a package for you, re-box it in their own packaging and then send it on to you. You then start to receive a package a day again.
At that point you start wondering, are they looking at the boxes, seeing they are from Amazon and then just deciding to leave them sitting around their warehouse for a while?
And the answer is yes, even though you're paying for the service, they also want whoever is sending the package to pay them to ensure that they get their package on time.
So while their may well be all sorts of agreements between all the different parcel companies about who will do what when, it's frustrating as an end-user that there's all these weird agreements that impact me. After all I'm paying for packages to be delivered, not 'packages that are offered by our partners'.
Note that this book predated the rise of streaming video providers by more than a decade. Back then, nobody thought that delivery of content over the Internet was this simple one-sided transaction. It’s a new view of the Internet that has arisen because it suits the interests of these very lucrative streaming businesses.
If UPS decided to change the way they internally route their packages, it doesn't mean that they can just brush off promised delivery dates because "it doesn't work how you think it does." I think that's the point the OP was making.
People like to pretend like internet access service is a promise to get your packets from point A to point B anywhere on the internet at a given speed. That is based on the fiction indulged by the software folks, who just think about getting bytes between a pair of sockets and don’t care how the internet actually works. But as explained in that Cisco article, interconnection and transit has always been distinct from access, and has been the subject of separate commercial negotiations between network operators. That reflects the technical reality that the internet is not a single network, but an agglomeration of private access and transit networks. The idealized software abstraction of the internet doesn’t define what it actually is. If you read the contract that defines what you're buying, it's not promising you that idealized abstraction.
> It's still the responsibility of the ISP to deliver connectivity to other hosts on the internet at their promised speed, regardless what is necessary to do so or how it operates internally
So they don’t have to deliver 1 gbps to every end point, they just need to “do a proper job” to make the bandwidth “usable.” But what does “proper job” mean? Historically, it has meant making reasonable efforts to reach interconnection agreements with other network operators. It has not meant that you’re required to upgrade your peering so that 50%+ of all your traffic can come from a single peer, at no cost to the other peer.
> Indulge in super speed and have a great online experience, whether you're browsing the Web, shopping, or streaming your favorite movies and shows.
Because if all I am paying for is access (to Xfinity's servers), then they cannot in any way guaranteed even being able to get to a simple blog.
They're marketing copy needs to be looked at.
The Internet is a bunch of interconnected networks. Any of those networks can largely have whatever conditions they want. That is what the Internet is. Do I think it should be different, yes, but a lot of people don't. Especially those arguing against things like throttling.
No, it's not like that at all. UPS makes it clear to the customer in advance what the cost is, how long it will take, and the customer agrees. Also, UPS is not held responsible for delays that they couldn't reasonably prevent (bad weather, etc).
It's also an exaggeration of the situation; it's essentially saying "If we can't achieve perfection, then we might as well not have any standards and we can't hold anyone accountable for anything." The fact is that those speeds are obtainable in most of the common situations that people actually need them, and the throttling only occurs out of unwillingness on the part of ISPs to negotiate agreements that would allow those speeds; it's greed and/or laziness, not physical or technical limitations.
At the end of the day what you are describing is some sort of regulated nationalized Internet backbone. Which would to a larger extent support such features. As it is now if you don't like the "mix" of access you are getting, you should change providers (which could of course be a problem, but that is another issue).
You're sold a "XXXMb/s connection to 'the internet'", which people take to mean "XXXMb/s of the thing that I want, whatever that is" and not "XXXMb/s connection to the edge of the network, and then you get what you get given."
Your post totally ignores peering agreements and interconnects that are key to understanding this issue.
Or when the firefighters get throttled because their unlimited data is consumed and your house goes up in smoke.
That you may have a faster connection to one provider than another doesn't indicate throttling, unless you can show that the network paths are essentially the same and the slower provider isn't the one restricting the data rate.
Certainly, almost any last mile congestion should affect all providers; but most content providers are going try to deliver content from as close to the user as possible; either through CDN boxes deployed inside ISPs networks if possible, or at a nearby place where the ISP or its transit partners peer. I'm aware that Netflix, Youtube, and Amazon all run their own CDN programs, so it's very likely that they're interconnecting to the ISP on separate connections, and without insider knowledge from the ISPs or the CDNs, it's impossible to know what the congestion looks like there. CDN installations also have capacity limitations, and take time to upgrade. It's not unreasonable to assume that some content providers are running their networks hotter than others.
In the years since the Internet has become mainstream, people seem to have forgotten the bad old days of downloading a large file, and trying to figure out which mirror would serve the file the fastest; which usually meant figuring out which mirror was both physically close, and well connected to the downloader's network.
I wonder if that's true, or this is simply a result of the peering agreements that content providers have engaged in since forever.
For those who don't know: if Netflix is colocated in the same datacenter(s) as CSPire, it's possible to "throw an ethernet cable over the cage wall" and make a direct connection to CSPire. This results in dramatically better bandwidth and latency, but requires significant up-front costs (it's also not possible for every provider to peer with every distributor, for logistical reasons).
As much as I believe in the concept of net neutrality, I feel like the conversation misses many of the technical nuances of the way the internet actually works. Peering is one example: you can have an entirely "neutral" internet, but still observe dramatic differences between one provider and another, based on your network location.
(Edit: now that I've read TFA, it's pretty clear that they're not talking about favoring providers, but rather, throttling entire classes of traffic. Which is a different discussion.)
> now that I've read TFA, it's pretty clear that they're not talking about favoring providers, but rather, throttling entire classes of traffic.
I must strongly disagree there. What kind of different class of traffic would give throttled traffic to different forms of on-demand video stream services? NBCSports, Netflix, YouTube Premium and Amazon prime. I don't see it so please name the four different classes of traffic those sites represent that a congestion system would use as identifiers.
There may be a small amount of packet loss at the time due to the ISP shaping their traffic to prevent congestion.
The higher the latency, the more the packet loss will affect the throughput.
The services you mention all use different CDNs, which may all have different latencies at the point of observation:
> NBCSports, Netflix, YouTube Premium and Amazon prime
NBCSports - I would guess Akamai, who deploys at IX and ISP Netflix - Usually Openconnect CDN devices at IX or ISP YouTube Premium - Google Global Cache deployed at IX and/or ISP (GCCs are deployed in most consumer ISPs, probably the best-distributed CDN at the moment). Amazon prime - Cloudfront, deployed mainly at IX, I am not sure if they have deployments in ISPs yet.
(I haven't tried to find more information on their approach, but https://dd.meddle.mobi/index.html seems to indicate that CDN issues shouldn't affect the way they measure, but until I do some checking on whether popular DPI platforms would be able to be fooled by this, and without them providing timestamped data, I'll have to test myself against DPI policies I wrote and I hope my ex-colleagues haven't changed too much ...)
The article makes it clear what they did, and what conclusions they drew:
"Using a previously established, peer-reviewed technique, the team conducted more than half a million data traffic tests across 161 countries. From this data, the team found that internet service providers are “giving a fixed amount of bandwidth—typically something in the range of one and a half megabits per second to four megabits per second—to video traffic, but they don’t impose these limits on other network traffic.”
Moreover, the data shown directly below that paragraph reveals little pattern in the provider/app throttling rates. The data shows mostly that video is being throttled across the board.
Could there be other explanations for this? Absolutely. Peering is just one of many plausible explanations.
And consumers have proved that they believed the first liars^H^H^H^H^Hmarketers who took this approach, so the rest, having tried the approach of being truthful and having lost market share, had to adopt the same approach ...
It is a super straight forward model, but obviously much more expensive to provide. I imagine most users would be happy to pay 1/10 of the cost to get a service that might drop to 10% performance in the worst case, as long as it doesn't happen too frequently.
A guaranteed 1 gig circuit is on the order of thousands USD month. An oversubscribed consumer fiber provider like Google Fiber is an order of magnitude cheaper because they can oversubscribe based on consumer traffic models that cover 99% of users.
Also keep in mind that your model isn't really straightforward at all. The ISP can only provide you guarantees within the lines they control. There could be any number of choke points at IXPs that mean the user can't reach site X at their guaranteed rate. So ISPs never even offer hard guarantees on connections to the wider Internet. The most you will be able to buy is dedicated circuits to their core, an IXP, some branch office, etc.
Are you okay with gyms selling more unlimited memberships than the number of machines they have?
Yes, but with a gym, you can switch if you keep seeing overcrowdedness. Not so with an ISP that has a monopoly.
Oversubscribing is a vastly preferable economic solution in this case. Everyone who can pay a small amount gets access to the network, and most of the time they get the bandwidth they need.
Selling a limited number of tickets at a massively increased price, or selling a massive number of tickets each with a dedicated data rate that was almost always idle are both strictly inferior solutions by very wide margins.
That throttling is essential to network management is undisputed. There were even carve outs in the old NN refs for it. The problem is if the “throttling” is not just limited video streams in general during peak times, but actually just limiting just certain services’ streams.
And this is the real problem.
The U.S. should require access-network sharing (at least for fixed-line, mobile seems to limitations here), like most other countries do ...
Then you wouldn't need to over-regulate how ISPs operate ... you could let the market decide.
Because that is not how wireless communication works. E.g. in LTE the user is scheduled both in frequency and time. The operator has a certain frequency spectrum that gives a certain maximum transmission rate (one user getting all frequencies and time).
The scheduling happens based on (amongst other things) signal to noise ratio and priority of each user and their traffic types (voice, data, etc). The goal of the operator is to maximize the overall capacity of the network. This is beneficial for the user as well.
I think the problem in the US is the lack of competition. Not net neutrality issues. In places where this works well you just change operator if you are not happy with your current one. This way the provider is free to experiment with how to run their network and everyone benefits.
I think it's because most NN advocates have a bias that regulation is a good thing, so of course regulating ISPs will be the right way to make the internet better.
However keep in mind that it is regulators who have granted the last mile monopolies to their crony firms and who have helped prevent last mile competition.
Notably one thing the FCC has done of late is to remove restrictions on reconfiguring wires on utility poles that benefitted incumbents. Now under the new rules, an ISP who is adding lines to existing utility poles is authorized to move/modify the equipment already on the pole when doing the installation.
Previously the rules allowed incumbent firms months to schedule a technician to go out to move the equipment around, which caused many delays for new firms trying to compete.
My internet service is pretty bad (XFinity) and Net Neutrality is not the solution, the solution is more ISPs to choose from. I'd much rather have three or four fiber providers to choose from than the (bad) choice between XFinity and AT&T. There are only two choices because of regulators, not in spite of them.
The latter is a much better solution. You don't seem to appreciate the realities at play here.
Which also, of course, explains why throttling it all the time is not necessary and certainly not optimal.
The economics of ISPs suck which means ISPs will suck pretty much as a rule.
How in the world would you guarantee that when you might not receive a strong/clear enough signal?
So the offer could look like this: "Up to 50Mbps, with 20Mbps offered at least 50% of the time between 06:00 and 22:00. Limited to 200GB/mo after which the speed is dropped to 1Mbps."
Also throttling and prioritizing types of traffic makes sense when you reach capacity. But if you suddenly find more capacity for someone paying extra by throttling others more or throttle indiscriminately and lift the limits for a price then you are not doing network management, you are are squeezing for money. And this should also be clearly stated when advertised and while doing the throttling.
As a customer paying the requested price I want some SLAs. I want to know what to expect for the service and be compensated for not getting it. I want to know realtime if my traffic or service is being throttled or not when I use it.
But this still suffers from exactly the same problem, just in a statistical way. i.e., what if someone's frequently in a place that has poor signal? How are you going to predict what percentage of the time they'll be in such a situation?
I don't intend to go into the technicalities because they're irrelevant. The point is they should offer a quality level at least "statistically".
Best effort is when your taxi driver tries to take you to the airport as fast as possible but traffic gets in the way and you're late. When he stops in a tunnel and requests more money because he has to drive through his lunch break, or else he'll just inch forward that's not best effort. That's what I said before: squeezing for money.
Reading the article helps: this is not network congestion, this is artificially limiting the speed.
And what is that context? Well, “unlimited” has always meant the opposite of time or usage limited service. I.e. you don’t pay extra for exceeding a certain usage. If you asked me in 1997 what "unlimited" means, I wouldn't have pulled out the dictionary and said "unlimited means 'not limited or restricted in terms of number, quantity, or extent.'" I would've said "that means you don't have to pay extra for AOL after your 20 hours is up." All this supposed confusion seems entirely contrived to me.
As per the links posted above:
> Santa Clara Fire paid Verizon for "unlimited" data but suffered from heavy throttling until the department paid Verizon more.
I'm an engineer so maybe I have a different definition for what an artificial limitation means. If you can pay for it and instantly get it then maybe it was there all along, just artificially limited. Physical limitations don't get removed with when your payment is cleared. And I'm paying for access to a network which has some intrinsic parameters. One of them is the number of towers, not the potential number of towers. I care about the actual physical speed and capacity describing the actual network.
We also seem to have different definitions of what "misleading" means since you obviously believe saying nothing about the limitations or their true nature while selling an untouchable maximum and even a patently false claim of "unlimited" is in line with "not resorting to dictionary definitions and ignoring context and history". Your explanation above is nonsensical. Nothing else gets judged by 1997 standards. VW should get a free pass for pollution in the context of "this was clean in 1997". EVs can claim they have unlimited range. Can't ignore that history. Maybe you also accept to be paid in 1997 dollars so when you charge a customer $80.000 they can just give you $50.000 and call it even. And some day you might even buy a 50TB HDD that will only have 50GB and some lawyer on the internet will tell you that in '97 people were happy with 5GB and you can remove that "physical limitation" with a paid FW upgrade.
The reason "unlimited" takes different meanings in these cases is that some countries are actually lead by lobbyists and when they say "jump" you ask "how high?". Not because you can ever reasonably argue "unlimited" means 20 hours because 1997. The proof? In many civilized countries such claims are illegal.
And I will say it again: actually reading the article would help you make some important distinctions.
Eschew flamebait. Don't introduce flamewar topics unless you have something genuinely new to say. Avoid unrelated controversies and generic tangents.
> saying nothing about the limitations or their true nature while selling an untouchable maximum and even a patently false claim of "unlimited"
> this is not network congestion, this is artificially limiting the speed
> they should offer a quality level at least "statistically"
> I want to know what to expect for the service and be compensated for not getting it. I want to know realtime if my traffic or service is being throttled or not when I use it.
> Informing the users properly is the the least they could do. Today they willingly mislead consumers
Changing the word "unlimited" is just one thing that would help clear up the confusion that you are getting unlimited data. That's one of many things they could be required to at least disclose (if not discontinue completely) in a clearer way especially since these usually involve multi-year contracts based on that misdirection. Basically exactly what's expected or required from most other companies' advertisements.
I honestly thought this would feel like common sense to anybody...
P.S. AT&T or Verizon have the following lines in their offers:
AT&T may temporarily slow data speeds when the network is congested
During times of congestion, your data may be temporarily slower than other traffic
This is disingenuous as it implies that they never do it unless the network is congested and that they don't carefully throttle only specific types of traffic from specific sources. That has been proven a false claim repeatedly.
This is fantastically well put.
https://drive.google.com/drive/folders/1yoBltbqtUWMN-cHMsHTd...
Funny thing, it was located on the same road where Tabasco sauce is made. I'm sure a tour bus passenger or two randomly caught me standing on top doing the (Brazilian) Christ the Redeemer (statue) pose.
Did you have to get a license to operate on 5.8ghz in your country? If so, what was the process like to get one with permission to offer commercial service?
Ah, sorry brainfart on my part, I didn't realize you were doing standard wifi. I always think of 802.11A(C) space as just 5GHz when it covers basically all of 5-6GHz. (For anyone interested: this has a nice list of channels https://en.wikipedia.org/wiki/List_of_WLAN_channels )
If I did it again, in a very rural area, I would look into licensed 4G/5G vs traditional wifi.
I was looking into this same thing (rural with low subscriber numbers) a few years ago. The costs of 4G and equipment for it were obscene compared to putting up ubiquiti equipment on 5g band.
- If you need to block streaming video because that's the only way your public WiFi on trains is going to work, iirc you can do that without any legal issues.
- If you zero-rate music services as a marketing stunt, go ahead, but then you shall zero-rate all music services. (Still seems annoying, as a music service, to go around the world and apply to every ISP offering this (i.e. it still makes it slightly harder for newcomers), but I guess it's not unreasonable.)
Is there a simple prosumer solution for this? My current path is leading me towards openwrt on a beefy router with some custom firewall rules.
On the access point side, if you want more granular control to handle congestion, Ubiquiti Unifi and their various mgmt platforms work well. I can't recommend their Security Gateway Appliance (frequent reboots), but the backhauls, APs and CPEs are solid. That will allow you to visualize utilization using a gui, and adjust limits per device/group easily. If you already have the APs, then a gateway device may be better.
I've found over the years, and in the datacenter, that running pfsense on server hardware is very reliable. You can run on a single board computer kit but the CPU doesn't hold up to heavier traffic/vpns etc, worse remote management.
Also, is there a way to charge for traffic beyond a certain quota? There’s a product called Sputnik with some firmware you can download for various router brands which lets you run a hotspot/captive portal with billing