But why did it take 18 months to realize that? You've got an issue that occurs consistently and is preventing customers from getting the service they're paying for (and are presumably in contracts so they can't even cancel without a major fight).
> They're too busy(!) to look for 10 customers' problem
So we're in agreement that this is unacceptable and resolving this problem has been grossly mismanaged and someone should be held accountable?
> doesn't monitor SNR levels because they're transient and doesn't provide meaningful data
This seems weird to me. You've got a nationwide network designed to run on old, poor quality wiring (the article doesn't specify whether it's DSL or coax, but it would apply to both). Wiring issues should be expected and automated processes in place to monitor it and quickly detect issues like this one.
> Not everyone use the standard equipment provided by the ISP
Can't this be collected at the DSLAM or CMTS level? Or in case of DSL, even at the PPP level? (Andrews & Arnold - a small DSL ISP in the UK - measures packet loss at that level and while it's not true SNR, it would be enough for these purposes as localized packet loss would pretty much always mean bad SNR or failing equipment)
Unless you had 50+% of the village calling up every morning for a week, it would not have been identified as a cluster event (and I suspect many ops orgs don't have the means to identify clusters anyway).
They concluded that something is fishy because normal procedures didn't fix anything. I'm not sure that this is a problem they diagnose 5 times a week.
> But the ISP should be able to tell whether their uplink is saturated (and actually upgrade it if it happens too often), so if they're consistently getting complaints around a certain timeframe and the uplink is not saturated during those timeframes then it must be something else.
Good luck escalating this problem from a village to correct people in a big-shiny-ISP(TM). They're too busy(!) to look for 10 customers' problem. I accept that 18 months in unacceptable but, this is the reality we're in. Support and Ops teams are always overworked because management thinks that "the load just doesn't justify more hands in that department".
> This is a problem then and should be addressed.
The backbone operators and ISPs doesn't monitor SNR levels because they're transient and doesn't provide meaningful data. If you want to do it, it's millions of connections. Also, Not everyone use the standard equipment provided by the ISP so, they cannot get info from all devices (e.g.: from mine). So, endpoint ops are complex if you're working on a WAN with random equipment.
Instead, using bandwidth graphs to find problems is more practical and reliable. Since the problem starts at 07:00, it shows like "They're not using their connections much, eh?" in the said graphs.
Disclosure: My workplace also happens to be the national academic network operator. I'm two hops away from a country-wide core router.