News for hackers. Stuff that matters.
Re:Facebook-owned sites were down by yoelo on Monday October 04, 2021 @03:16PM (#28750850)
Seriously? Is that how it works?
Re:Facebook-owned sites were down by cranekam on Monday October 04, 2021 @03:41PM (#28751147)
No. A network like Facebook's is vast and complicated and managed by higher-level configuration systems, not people emailing patches around.

If this issue is even to do with BGP it's much more likely the root of the problem is somewhere in this configuration system and that fixing it is compounded by some other issues that nobody foresaw. Huge events like this are always a perfect storm of several factors, any one or two of which would be a total noop alone.

Re:Facebook-owned sites were down by KuiN on Monday October 04, 2021 @04:13PM (#28751481)
The Swiss cheese model of accidents. Occasionally the holes all align.

https://en.wikipedia.org/wiki/Swiss_cheese_model

Re:Facebook-owned sites were down by soneil on Monday October 04, 2021 @05:53PM (#28752563)
The fun part of BGP is they apparently make a lot of use of it within their network, not just advertising routes externally.

https://engineering.fb.com/2021/05/13/data-center-engineerin...

(and yes, fb.com resolves)

Re:Facebook-owned sites were down by weisk on Monday October 04, 2021 @03:28PM (#28750992)
No, the backbone of the internet is not maintained with patches sent in emails.
Re:Facebook-owned sites were down by chiluk on Monday October 04, 2021 @03:40PM (#28751139)
You are very wrong about that ;) https://lkml.org/
Re:Facebook-owned sites were down by chiluk on Monday October 04, 2021 @03:40PM (#28751135)
You are very wrong about that https://lkml.org/
Re:Facebook-owned sites were down by cbarrick on Monday October 04, 2021 @04:14PM (#28751502)
Clearly you and the person you replied to are talking about very different things.
Re:Facebook-owned sites were down by _joel on Monday October 04, 2021 @03:44PM (#28751184)
I think the sub-comment is confusing the linux kernel with BGP.
Re:Facebook-owned sites were down by nacs on Monday October 04, 2021 @03:55PM (#28751276)
In a way, the Linux kernel does power the "backbones of the internet".
Re:Facebook-owned sites were down by _joel on Monday October 04, 2021 @03:59PM (#28751332)
There are a hell of a lot of non-linux OS's running on core routers, but yes, in a way. However BGP isn't via email.
Re:Facebook-owned sites were down by dsr_ on Monday October 04, 2021 @08:31PM (#28754139)
On the other hand, I and my office mate at the time negotiated the setup of a ridiculous number of BGP sessions over email, including sending configs. That was 20 years ago.
Re:Facebook-owned sites were down by NexRebular on Monday October 04, 2021 @05:08PM (#28752102)
luckily not... would be absolutely terrible to have the backbone only on linux
Re:Facebook-owned sites were down by dsr_ on Monday October 04, 2021 @08:32PM (#28754150)
Interoperability and a thriving ecosystem are necessities for resiliency.

Note that resiliency and efficiency are often working against each other.

Re:Facebook-owned sites were down by chiluk on Monday October 04, 2021 @03:39PM (#28751128)
Re:Facebook-owned sites were down by john37386 on Monday October 04, 2021 @03:19PM (#28750889)
I don't know. I doubt. It's just funny to think that you need email to fix BGP, but DNS is down because of BGP. You need DNS to send email which needs BGP. It's a kind of chicken and egg problem but at a massive scale this time.
Re:Facebook-owned sites were down by boshomi on Monday October 04, 2021 @03:37PM (#28751102)
Sheera Frenkel:

    Was just on phone with someone who works for FB who described employees unable to enter buildings this morning to begin to evaluate extent of outage because their badges weren’t working to access doors.
https://twitter.com/sheeraf/status/1445099150316503057
Re:Facebook-owned sites were down by _joel on Monday October 04, 2021 @03:42PM (#28751155)
You'd think they'd have worked that into their DR plans for a complete P1 outage of the domain/DNS, but perhaps not, or at least they didn't add removal of BGP announcements to the mix.