People talk about how kubernetes is overkill for many people, but it gives you rolling deployments for your applications out of the box.
There's also the "slippery slope" argument. A 0 downtime migration means 0 downtime. A 5 minute migration creeps up to a 30 minutes migration occasionally, and then regression to the mean happens, causing standard migrations to be 30 minutes.
At least where I'm working downtime is acceptable because our customer base is composed of professionals in north america that work 9-5 thus taking the DB offline for five minutes at midnight PST has essentially no cost to our client base. To contrast that, spending three months developing a rollout plan costs our company dearly in competitive advantage.
I agree that it's always worth examining from both sides but I also think that 0 downtime migration is both a shiny interesting problem and a point of pride for developers that can cause a company to vastly over invest in a complex technical solution to a problem they never had.
Allowing maintenance windows means you can do things in a much simple manner. Need to take a snapshot of a DB without running into issues with a production system adding data? Sure, go ahead, you just saved two months.
The principle of something like zero downtime or switching a vendor out to save 20% may not stack up against the reality in opportunity costs, employee burnout, and customer turnover.
My company is reorganizing and so everyone is trying to settle old philosophical debates during The Churn. But everyone is overextended and a number of us have started pushing back. Just the meetings to discuss changing vendors might push the break-even point out 6 months, before you even get into implementation costs. The more time we spend working on infrastructure instead of customer-facing features and bugs, the more likely those customers are to wander off.
I'm not saying there's no downside, I'm asking against the downside of engineering cost. And that itself carries risk of failure when you go live. It's not guaranteed.
[1] AWS is so ubiquitous though, that half the internet would be affected so it makes it less individually noticeable.
I hope you have everything redundant, including the CEO
Made me laugh, I'll sure reuse it!
One of these things is not like the other.
This goes both ways -- there have been many AWS outages which have not affected us. I hear what you're saying, but we've had only one instance of extended (hours) downtime in the last 20 years.
For the app servers, saltstack is used and we synchronize that repository with what is needed to reproduce a production environment in AWS.
Obviously we'd have to provision servers, etc, but it's all possible in a worst-case scenario.
I looked at the company, and while they seem rather large, they're not Netflix or AWS.
I imagine they need to be up for people to be able to check in, etc. But they could just block out the planned maintenance as check in times far in advance. I'm sure there's a million other edge cases but those can be thought out and weighed against the engineering effort.
Don't get me wrong, this is very cool.
But I wonder what the engineering cost was. I'd think easily in the hundreds of thousands of dollars.