I work for a ~billion dollar organization in risk management. If developers can't certify to our team that you can pull your app off of $cloud_provider and have it running on prem or in another cloud provider in a few weeks, it doesn't go to a cloud provider. We'll assist with their IaaS/PaaS tooling selection, infrastructure orchestration, container lifecycle process, and other ancillary needs to ensure this requirement can be met.
The issue isn't survivorship bias, it's risk management, business continuity, and cost management. It's entirely possible your organization doesn't place a high dollar value on these risks, but that's a business decision each organization needs to make for itself as a whole, and its products and services individually.
A) I start a company, get some investors and sell the company to a larger company that has the resources to either keep maintaining it or change out the infrastructure (Instagram). Result: I profit
B) I start a company, gain product market-fit, become successful and I have the resources to gradually move my infrastructure when the need arises: Result - profit.
C) I worry about "vendor lock-in" from day one, spend a lot of time and resources on my own infrastructure and run out of money and never survive long enough for A or B.
I know which one I'm going to choose.
And as far as not "needing the complexity", the entire purpose of going to AWS is to avoid the complexity and up front cost of providing your own infrastructure.
The HN darling Dropbox started out on AWS because it was faster and then when they grew they built there own infrastructure.
Netflix built their own infrastructure and decided that infrastructure wasn't their core competency and went entirely to AWS.
There is no one size fits all.
The reality is you'll need to do it when you are tight on resources and you're stuck rebuilding what has become a very complex infrastructure (because you spent the last 5 years hooking into every proprietary bell and whistle) from scratch - with no internal expertise because you didn't start early when it was still a tractable problem.
I'm not saying you need to choose between AWS or running on your own physical hardware. But I am saying you don't want to lock in to the proprietary parts of AWS, which do tend to add lots of complexity to your infrastructure because they make that complexity easy to use.
Spinning up some ubuntu ec2 servers is fine - you can move those anywhere easily.
I've worked for a company that had two issues - badly written software and a badly architected AWS environment. Guess which one was easier to fix?
All of the companies that tied their fate to Webforms, Ruby (Twitter), or some other deprecated tech have far bigger problems than someone who tied themselves to AWS.
As far as AWS what complexity is that?
Databases - all of their databases are compatible with standard databases except for DynamoDB. DynamoDB sucks anyway. If I wanted a managed NoSQL database, I would go with one from Mongo.
Messaging - You usually end up sending and reviewing messages from one central place in your code, changing your messages system out is pretty simple.
Caching - ElastiCache exposes either a Memcached or RedisCache client.
Load balancing - the only thing special you do as far as software is have a health check endpoint. I used the same endpoint that I used on prem with Consul+Fabio.
ElasticSearch -same as using your own.
Autoscaling - I've got nothing. I've never tried doing autoscaling on prem.
Codepipeline/CodeBuild/Code Deploy - the triggering event is nothing more than a standard Web hook from your git repo of choice, Code Build is a preconfigured Docker Container that executes commands based on a yml file. I use a Python Lambda function for deployment, but I could run that in anything.
Even Lambda if written correctly - your handler should be skinny and treated like a Controller - shouldn't be that hard to port over.
Fargate/ECS - orchestration for standard Docker containers.
Parameter Store - yeah I could (and have) set up a cluster of Consul/Vault servers but why would I instead of using a service that ties in security roles and encryption with everything else?
Load Balancing - I've also set up my own Consul+Fabio clients on web servers/app servers for load balancing and service discovery. Not trying to do that anymore either when I can configure an ALB and an autoscaling group.
The software wouldn't be the hard part to move over. Even if we ran everything in EC2 instances, the hard part would be duplicating multiple geographically redundant VPCs in multiple locations in the US and an environment in Asia that was close to the outsourced developers.
Netflix went all in on AWS and much of their tooling is tied tightly into AWS. If Netflix can trust its entire business to AWS, I think anyone can.
This is a terrible argument. Netflix is simply a media streaming company, with very little importance compared to other businesses. Most of its data is served from its own CDN!
Check out a few of Netflix's blog posts, AWS re-invent presentations (available on YouTube), the podcast interview, I posted earlier, and thier open source project. Netflix uses a lot of AWS services.
Edit: just realized I duplicated scarface74’s comment. My bad!
What you're talking about here is survivorship bias -- yes, indeed you will find people paying big sums to migrate off of AWS for whatever reason. But they're still around! Unless of course you assume using AWS (or any vendor for that matter) provides no competitive advantage whatsoever, even in the short term.
The specific category of risk you are talking about here are tech ventures that fail because of a specific sequence of events:
- They choose AWS
- They, miraculously, create a successful business on AWS
- Some existential risk emerges due to their use of AWS
- They either fail or cannot afford to move off of AWS
- This singular decision causes their business to fail catastrophically
- And, they would have been successful had they not chosen AWS in the first place
This is a very specific path to failure and is highly unlikely relative to the more common cases of "they never found product-market fit" or "they had a toxic culture" or "they decided to build everything themselves and ran out of runway."