Re:Cloudflare outage on December 5, 2025
by hrimfaxi on Friday December 05, 2025 @10:57AM
(#46163001)
Having their changes fully propagate within 1 minute is pretty fantastic.
[ Parent
| on HN ]
Re:Cloudflare outage on December 5, 2025
by denysvitali on Friday December 05, 2025 @11:10AM
(#46163226)
This is most likely a strong requisite for such a big scale deployment if DDOS protection and detection - which explains their architectural choices (ClickHouse & co) and the need of a super low latency config changes.
Since attackers might rotate IPs more frequently than once per minute, this effectively means that the whole fleet of servers should be able to quickly react depending on the decisions done centrally.
Re:Cloudflare outage on December 5, 2025
by chatmasta on Friday December 05, 2025 @11:07AM
(#46163181)
The coolest part of Cloudflare’s architecture is that every server is the same… which presumably makes deployment a straightforward task.
Re:Cloudflare outage on December 5, 2025
by jamesog on Friday December 05, 2025 @02:55PM
(#46166447)
The bad change wasn't even a deployment as such, just an entry in the global KV store https://blog.cloudflare.com/introducing-quicksilver-configur...
Actual deployments take hours to propagate worldwide.
(Disclosure: former Cloudflare SRE)
Re:Cloudflare outage on December 5, 2025
by reassess_blind on Friday December 05, 2025 @05:16PM
(#46168097)
Why wasn’t the rollback fixed within the second minute after they saw the 500s?