Additionally to say that building something outside of the cloud ensures you're cloud agnostic is too simplistic. There are many more shops that would like to move to the cloud to reduce their costs, but can't due to on-site lock-in. They then run into much more expensive problems around expanding datacenters, network or power limitations, or staffing to maintain physical machines which have real world MTBFs you have to do something about when they fail.
I think the issue isn't as much lock-in as much as apps being designed poorly, reliant on a specific feature that's not necessary, but then means their app will only work in one cloud as a result. However in the cases where there is no way around that feature, then you're getting a huge benefit of an instant solution so you can focus on your product rather than infrastructure. And if that cloud does provide a single killer feature you need, then just run that feature there.
In my experience, the difference in personnel between a service built in AWS versus one built in datacenters isn't wildly different. If you have a five person team for the former, you might need a six or seven person team for the latter. You aren't dealing with hardware and vendor problems in AWS necessarily, but it's also not as simple as 'just run your app and forget the rest' (at least once you start to scale up).
Again though, this is if you're smart about how you build the datacenter. There are a lot of tricks of the trade and a bunch of gotchas that aren't necessarily apparent until you've experienced them for yourself. Hiring experience here is the key.
While you don't necessarily want to limit yourself to the lowest common denominator of cloud providers or your own hardware, any time you make use of something unique you need to make a decision. Is it worth getting locked in? What will you do if that solution goes poof in the night?
To use a real recent example: what do you do if AWS runs out of X1 instances and you really really need some more right now to do your work? It sucks to get into a situation where you have no control over the outcome and even money can't fix it, either in your own datacenter or in AWS.
Also, AWS is not problem free. When low level issues (like a firmware problem) strike, which they sometimes will, you often cannot identify or fix the issue yourself. AWS instances still run on that same shitty hardware, and while your odds are better that Amazon figured out most of the issues before you encountered them, your odds are also worse that they tested your particular special use case.
So you can wind up playing the 'debug by support ticket' game. Check everything on your end. Check your deployment times for correlations to the problem. Finally give up and reach out to AWS support. Insist that the issue isn't on your end. Find out that the issue can't be resolved until their deployment next week. It's all understandable given the scope of what Amazon is doing, but they will never be as incentivized as you are to solve your problem.
Yes, you still have to open tickets, and yes, they still have problems that suck up a lot of time, however they spend the brunt of the time burning folks out fixing those issues, not you. If you talk to folks working with physical systems, that consumes their mindshare to the point that they're not focusing on their core product anymore.
All I can say is, it isn't always the case and it doesn't have to be with the right organizational and design decisions.
Or in renting dedicated servers.
I’ve done some checks for the same performance between colocating, dedicated, VPS, and stuff like container engine + cloud databases, and with each level of abstraction the cost per month usually goes up by an order of magnitude.
It makes sense that each layer of abstraction costs dramatically more, because these services aren't vertically integrated -- they operate as value adds on top of other services. Heroku runs on AWS who buys server hardware and network hardware from vendors and puts it in racks that sit in buildings operated by datacenter providers who supply power from the electric company.
So when you run something at the abstraction layer of a Heroku dyno, you're contributing to the profit margins of at least:
1. The actual service you use (Heroku, and Salesforce)
2. Their instance provider (Amazon)
3a. (maybe) Their server hardware manufacturer (HP / Dell / Supermicro)
3b. (maybe) Their network hardware manufacturer (Cisco / Juniper / Arista)
4a. Their server ODM (Quanta / et. al.)
5a. Their server components manufacturer (Intel / Hynix / Samsung / HGST / Mellanox / Adaptec / Avago / etc.)
5b. Their network ODM or ASIC designer (Avago)
6. (maybe) Their hosting provider (Peer1 or someone)
7. (maybe) Their retail datacenter provider (Equinix / CoreSite / etc.)
8. Their wholesale datacenter provider (DuPont Fabros / Digital Realty / etc.)
9. Their suppliers and contractors (electrical contractors / Square D / security guards /
10. The electricity and the taxes paid to the municipality
Each of these companies intends to make a profit on the services they provide to the company up the food chain, and that all gets bundled into what you pay.Sure there are economies of scale here that you can't get, and smart companies are trying to eliminate some of these layers -- but that doesn't necessarily mean you are paying less.
The margins increase as you go up the stack. Electric companies make peanuts for margins (8-10%) compared to a company like Mellanox that makes $1200 network cards (20+%).
But who cares about selling you a $1200 network card when you can charge people $100/mo to use a dyno?
It also depends on how your service is positioned to grow. If you have any sort of reliable, foreseeable growth then you can save money by building infrastructure yourself. You are paying for Amazon's overhead to allow you to spin up any instance anywhere at any time (and there's no guarantee it will really be there) -- even if you're buying hardware 6 months ahead of when you actually need it, as long as you're growing and not missing the mark too badly you can do a better job on cost.
Most importantly: are you planning for success? If you sign a giant customer, will it help you grow your margins by getting you to better economies of scale with your vendors and your expenses? Or will it crush you because now your burn rate just doubled? There are economies of scale available with AWS, but they aren't great and you have to be pretty huge to get anything meaningful.
Two of your examples stuck out to me:
Dropbox was never entirely run in AWS. That complicated things operationally to a degree, but ultimately made it a lot easier to move pieces of the service in or out of AWS as it made financial and technical sense.
Pinterest was built in AWS. Since they waited so long to start seriously considering alternatives, they're in a tough spot. Migrating out will require a substantial capital commitment (not to mention engineering and product commitments), and they'll have to pay for both environments during what is liable to be a lengthy cutover period. You have more resources to leverage, but the scale of the problem just gets harder over time.