Re:Post Mortem on Cloudflare Control Plane and Analytics Outage
by nanankcornering on Saturday November 04, 2023 @02:40AM
(#38138708)
At what time were you notified Matt?
[ Parent
| on HN ]
Re:Post Mortem on Cloudflare Control Plane and Analytics Outage
by eastdakota on Saturday November 04, 2023 @02:43AM
(#38138721)
Of the incident? Someone on my team called me about 30 minutes after it started. It was challenging for me to stay on top of because it was also the same day as our Q3 earnings call. But team kept me informed throughout the day. I helped where I could. And they handled a very difficult situation very well. That said, lots we can learn from and improve.
Re:Post Mortem on Cloudflare Control Plane and Analytics Outage
by chaz6 on Saturday November 04, 2023 @03:40AM
(#38138952)
What I find bizarre is that the Cloudflare share price jumped when the outage happend!
Having read the post mortem, I do not think it could have been handled any better. I think the decision to extend the outage in order to provide rest was absolutely correct.
I always enjoy reading these reports from Cloudflare as thare are the best in the business.
Re:Post Mortem on Cloudflare Control Plane and Analytics Outage
by eastdakota on Saturday November 04, 2023 @03:44AM
(#38138962)
I was surprised we didn't get a single question about it from an analyst or investor, either formally on the Q3 call or on any callbacks we did after. One weird phenomenon we've seen — though not so much in this case because the impact wasn't as publicly exposed — is that investors after we've had a really bad outage say: "Oh, wow, I didn't fully appreciate how important you were until you took down most of the Internet." So… ¯\_(ツ)_/¯
Re:Post Mortem on Cloudflare Control Plane and Analytics Outage
by dboreham on Saturday November 04, 2023 @04:20AM
(#38139094)
There's a class of investor (and their trade bots presumably) that sees outrage over a service outage as proof the provider is now mission critical, hence able to "extract value" from the market.
Re:Post Mortem on Cloudflare Control Plane and Analytics Outage
by mercurialuser on Saturday November 04, 2023 @05:08AM
(#38139290)
Did you rebuild all the server from scratch?
Re:Post Mortem on Cloudflare Control Plane and Analytics Outage
by iAMkenough on Saturday November 04, 2023 @07:26AM
(#38140045)
Coincidental timing?