News for hackers. Stuff that matters.
Re:Cloudflare outage should not have happened by nine_k on Wednesday November 26, 2025 @03:08PM (#46061788)
Indeed, I never worked at Cloudflare. Still I have some nebulous idea about Cloudflare, and especially their scale.

Systems can do worse things than crashing in response to unexpected states, but they can also do better to report them and terminate gracefully. Especially if the code runs on so many nodes, and the crash renders them unresponsive.

Blue/green deployment isn't always possible, but my imagination is a bit weak, and I cannot suggest a way to synchronously update so many nodes literally all over the internet. A blue/green deployment happens in large distributed systems willy-nilly. It's better when it happens in a controlled way, and the safety of a change that affects basically the entire fleet is tested under real load before applying it everywhere.

I do not even assume that any of Cloudflare's code was ever shipped with the "move fast, break things" mindset; I only posit that such a mindset is not optimal for a company in the Cloudflare's position. Their motto might rather be "move smooth, never break anything"; I suppose that most of their customers value their stability higher than their speed of releasing features, or whatnot.

Starting with questions is a very right way, I agree. My first question: why calling unwrap() might ever be a good idea in production code, and especially in some config-loading code, which, to my mind, should be resilient, and ready to handle variations in the config data gracefully? Certain mechanical patterns, like "don't hit your finger with a hammer", are best applied universally by default, with the rare exceptional cases carefully documented and explained, not the other way around.

Re:Cloudflare outage should not have happened by kalkin on Wednesday November 26, 2025 @06:37PM (#46063573)
I appreciate that this comment is much less prescriptive. I don't think I disagree with you about any general best practices here (although I do think unwrap can be fine when you can locally verify the error or nil case is unreachable but proving that to the compiler is impractical.)