Consul is awesome when it works, but when it breaks it can be hell to get it working again. thankfully it usually works fine. I only had 1 outage and it fixed itself after restarting the service.
Interestingly, reading this remind me of a HashiCorp Nomad marketing piece [1]:
> "We have people who are first-time system administrators deploying applications, building containers, maintaining Nomad. There is a guy on our team who worked in the IT help desk for eight years — just today he upgraded an entire cluster himself."
I was always thinking "but what if something goes wrong? just call HashiCorp engs?" :p
Just the author of the previous post relying on these types of services is something that can keep me up at night.
Consul (and Vault) for sure a complex softwares that 99% of the time "just work", but when they fail they can fail big time, I concur. But calling it a mortal risk seems a bit far fetched in my opinion.
Quite frankly relying on consul scares the shit out of me. There are so few guarantees and so many pitfalls and traps that I don’t sleep well. At this point I consider it a mortal risk.
That applies to vault as well.