Re:Fly.io outage, recently deployed apps down, deployments d...
by namaria on Sunday March 19, 2023 @06:43AM
(#35217729)
"Should have self healing" can be expanded into "should have systems to address the underlying system failure modes", which starts to shed some light into why distributed systems will always run into failure modes.
Also, "self-healing" isn't really one thing. There are hundreds of different problems that can take out such a cluster, and every single one of them needs its own "self-healing" mechanism. These systems are literally the most complicated kinds of systems.