Two of our local mobile operators are experiencing issues with phone calls due to network overload.
https://twitter.com/claroelsalvador/status/14450819333319598...
can't use the phone network to place a call b/c of fb-errors clogging the pipe
https://news.ycombinator.com/item?id=25803010 Signal apps DDoS'ed their own server
Second comment was saying there is no point using Signal if they are down during 2 days. Only a few hours for FB yet but curiously nobody is saying the same :)
Let's hope it's done wrong.
And I'm glad you can feel smug about combating smuggness, because obviously the consequences of some social media and chat apps being down can't be measured but by anecodotal stories of some unrelated issue like being locked out, not about their general societal impact, shady practices, contribution to disinformation and data mining, etc. Who's being self centered now?
If anything, the lesson here is to not depend on some single, centralized, channel, for such communications (e.g. to get your AirBnB key). Now I also feel smug for always giving 2-3 alternative ways in cases contacting someone/someone contacting me is crucial...
It's not like what the world lacks in 2021 is communication channels. One can use land phone, mobile phone, SMS, email, and 200 other alternative IM outlets...
[1] https://datatracker.ietf.org/doc/html/rfc2308#section-7.1
I think you missed the idea that the FB outage created a really heavy DNS load on other people's DNS servers.
It's not so simple to cache misses - you don't know if it's a real miss or some kind of error.
For example if Facebook cached the miss, then even when they are back up nothing would connect.
This tends to be beneficial regardless of the root cause.
edit this is especially useful for handling search/query misses as a query with no results is going to scan any relevant indexes etc. until it is clear no match exists meaning a no results query may take up more cycles than a hit.
In the past I was the architect of a top 2000 alexa ranked social networking site, data synchronization delays were insane under certain load patterns high single low double digit second write propagation delays.
Even if you assume Facebook's own apps have reasonable failure logic, there's all kinds of third-party apps and devices integrating with their API that probably get it wrong. Surprise botnet!
But if the request does not resolve there’s no caching, the next request goes through the entire thing and hits the server again.
I don't know what facebook's DNS cache expiration interval was, but assume it's 1 day. Now multiply the load on the DNS that those facebook users put by whatever polling interval the apps use.
And then remember what percentage of internet traffic (requests, not bandwidth) facebook, whatsapp, and instagram make up.
It's kindof beautiful.
Anecdotally, it also seems to be draining the batteries of those devices with all of those extra queries. At least that seems to be what's happening on my wife's phone.
Could this bring down the whole internet for a while?
What has just happened, and it can't be coincidence, is that I lost internet connectivity about 1 hour ago, and had to reboot my Cable Modem to get it back.
Unfortunately I'm not sure what the default DNS on the modem points to..
Looking at queries for graph.instagram.com, it looks like there are roughly 20 attempts in a sequence before it gives up.
All in all, this could probably explain doubling of the DNS traffic. But the sample is rather small, so take it with a grain of salt.
If you aren’t using exponential backoff algorithms for your reconnect scheme - you should be!
I have a device in the field, only a few thousand total, but we saw issues when our shared cloud would go down and everyone hammered it to get back up.
It's not crazy; people are panicking over Facebook, Instagram and WhatsApp being down and they keep trying to connect to those services. I mean I would panic too if I were social media junky.
>Now, here's the fun part. @Cloudflare runs a free DNS resolver, 1.1.1.1, and lots of people use it. So Facebook etc. are down... guess what happens? People keep retrying. Software keeps retrying. We get hit by a massive flood of DNS traffic asking for http://facebook.com
https://twitter.com/jgrahamc/status/1445066136547217413
>Our small non profit also sees a huge spike in DNS traffic. It’s really insane.
https://twitter.com/awlnx/status/1445072441886265355
>This is frontend DNS stats from one of the smaller ISPs I operate. DNS traffic has almost doubled.
https://twitter.com/TheodoreBaschak/status/14450732299707637...