Re:Cloudflare outage on November 18, 2025 post mortem
by mike_hearn on Monday November 24, 2025 @05:19AM
(#46032389)
OK, so the issue is frameworks not catching panics and logging proper stack traces? Very cool that you made a library.
https://doc.rust-lang.org/std/panic/index.html
An uncaught exception in C++ or an uncaught panic in Rust terminates the program. The unwinding is the same mechanism. I think the implementation is what comes with LLVM, but I haven't checked.
I was also a Google SRE, and I liked the stacktrace facilities so much that I got permission to open source a library inspired from it: https://github.com/bombela/backward-cpp (I know I am not doing a great job maintaining it)
At Uber I implemented a similar stackrace introspection for RPC tasks via HTTP for Go services.
You can also catch a Go panic. Which we did in our RPC library at Uber.
It would be great for all of that to somehow come ready made though. A sort of flag "this program is a service, turn on all the good diagnostics, here is my main loop".