While they haven't actually explicitly said that, and certainly my employer hasn't heard that message, our AWS rep has repeatedly and loudly said that all of our planning should incorporate failure starting from the smallest (like an individual instance) up through a service in an AZ to a service in a region or a complete AZ outage, to a complete regional outage or global service outage to a global provider outage.

So, the message is implicitly there.

> shit that supposedly work correctly at the time.

I do love me a well-formed customer query.

Yeah, it's supposed to work. I get paid more it if does. But in exceptional circumstances, it won't. And in more exceptional circumstances your alerts will fail too.

As an admin what do you do with the 90% of your time that isn't actively fixing something if not planning for how to fix the next thing? We did this years before cloud meant anything other than prepare for floods.

> As an admin what do you do with the 90% of your time that isn't actively fixing something if not planning for how to fix the next thing?

Not every cloud customer has a dedicated admin team, some even don't have any dedicated developer teams. They simply wanted a website done, contracted someone and they uploaded some HTML to a cloud instance.

Sure, if you're a large company, it makes sense to have redundancy. If you're running a small company with some esoteric webshop to serve AFK customers, it makes less sense.

If it didn't take a team to design what you use then it probably won't take a team to design a fallback. But the difficulty doesn't excuse not doing it, if you don't have a fallback for whatever you pick, pick something simpler.

Ideally we keep stuff running, but I don't want to be the kind of person who just tells you that we've got it. You need to know your airbag might fail so that you take your seatbelt seriously.