News for hackers. Stuff that matters.
Re:Cloudflare outage on November 18, 2025 post mortem by antihero on Wednesday November 19, 2025 @07:27AM (#45978766)
The unwrap should be replaced by code that creates enough alerting to make a P0 incidident from their canary deployment immediately.

OR even, the bot code crashing should itself be generating alerts.

Canary deployment would be automatically rolled back until P0 incident resolved.

All of this could probably have happened and contained at their scale in less than a minute as they would likely generate enough "omg the proxy cannot handle its config" alerts off of a deployment of 0.001% near immediately.

Re:Cloudflare outage on November 18, 2025 post mortem by vlovich123 on Wednesday November 19, 2025 @11:27PM (#45988959)
Agreed - a big question why the file wasn’t test driven in staging and progressively rolled out. And also what alerting was missing within FL2 that they couldn’t pinpoint the unwrap instantly.