It is common for ISP to host instances of the Google Global Cache (GGC, see https://peering.google.com/) which are used for many Google services, most importantly YouTube.

In fact, in many cases Google itself "suggests" to ISP that they host a few GGC servers.

They are directly monitored by Google, and the ISP has basically no say in how they are run. Capacity is managed by Google directly.

Yep, Virgin Media did this in the UK and messed it up badly. Every day at 6pm YouTube would stop working until the following morning.

It appeared to work by inspecting DNS packets and replying with overrides if necessary. I didn’t like it but I could understand that.

What I did not agree with was the fact that this also happened for other DNS services. Google DNS and OpenDNS both experienced the same issue, as did a few other “famous” DNS providers. Random little ones wouldn’t return the caching servers, and also enabling encrypted DNS for Google/OpenDNS would stop it happening too. I’m fairly sure it was some badly thought out deep packet inspection.

Again, it's probably _not_ Virgin's fault or responsibility. Capacity planning is handled by Google directly.

Also, I think that virtually every ISP with more than a few tens of thousands users is hosting a GGC instance nowadays (and a Netflix OpenCache, etc. etc.).

Nowadays, the vast majority of the transit&peering of ISPs is not going to the Internet, but to a few racks of local caching servers managed by OTT operators.

Bandwidth-wise, at least in prime time, the Internet is much less connected/realtime than people think :)

I would suggest that the deep packet inspection is Virgin's fault. Perhaps it was Google under provisioning the CDN inside VM's network, but I would suggest that's also probably Virgin's fault in part, as they will likely have more network information available to them that may have suggested network congestion, but that was not acted upon.

As someone else suggested, this is likely part of our GGC program. If you can give me info on which ISP + Geographic region you're in, I can take a look and see if there's anything in our logs to indicate a problems; you can email me details at myusernamehere at google dot com with details, since I don't routinely check Hacker News.

If you can reproduce a bad experience, and right click on the player, click "Get Debug Info", and share that result, it's the most helpful thing for us to dig into problems.

FYI, something that makes my Hacker News experience much nicer is email notifications when my comments are replied to.

Get your own at http://hnreplies.com/ - built by Dan Grossman

>My isp got the brilliant idea of rolling their own YouTube cache servers

Are YouTube videos not served using https? Sounds like it's google/YouTube servers but they're underprovsioned

The YouTube cache is probably there not to increase performance for you, but to reduce ISP cost (ISP pays for all data received from outside of their network)

Even for data not delivered via their cache they'll get free peering with Google. But having cache servers at several locations reduces the capacity they need to peering points. E.g. in the UK, most traffic gets peered in London so having cache servers in Northern England reduces the bandwidth they need for their internal network within the country.

I don't think ISPs pay much for peering in Europe.

That's a strange way of looking at it. ISPs take all sorts of actions to both increase performance to customers, and to reduce their costs.

A competent ISP operating in good faith will execute and expand peering agreements when it makes economic sense to do so.

An ISPs #1 goal is to generate money.

When there's a competition this often equates with making customers happy so they will stick with the ISP and perhaps more will switch from competitors.

When there's no competition and customers have no choice who to use it's purely all about squeezing as much money as they can. This means raising prices for the service and reducing cost by paying less to others services even if it would reduce quality of service. The goal is to have service that's good enough to keep people still use them (I mean there is price point and/or quality that people would just not to have service at all).

If a specific action that ISP with monopoly does and it improves quality for customer that's just a coincidence.

> A competent ISP operating in good faith

That's probably the funniest thing I've read all year