Core banking, batch processing, highly sensitive data stores? Probably not great candidates for public cloud consumption.
Web properties and services which don't rely on said functionality? Absolutely great candidates.
And the reality is, whether the IT guys like it or not, developers inside of these orgs are consuming cloud because getting a VM in a traditional sense takes forever (for good reason.)
As a result, we're seeing a shift in the industry where the idea of large corps / financial institutions / government are being pragmatic about the idea of a 'hybrid cloud.'
The reality is, adoption of cloud in the enterprise is growing, not shrinking.
Say you could be paid $100 p/h at some other company, but instead your start up is paying you $5 p/h because that is all it can currently afford.
Say you can save yourself a weeks work by going with one hosting option that is twice as expensive but the total additional cost would allow you to work for $5 p/h for an additional two weeks then that isn't a great trade because the "value" of that labor is both unknown and irrelevant at this point. It might be $0 p/h or it might be $10000+ p/h.
So even at $5/hr I can't justify doing this work myself at this point, since I can get the infrastructure for my minimum viable product going for free. I can and should spend my time focusing on developing the features.
This is honestly why I've gone with PaaS - mostly Heroku - for several months now when deploying a new application. Why on Earth developers do anything other than working on the core features of their program I don't know. All of the things you need to set up - tesing pipeline, and containerized, automatic deployment, load balancing, databases - are now available as cloud services. There is absolutely no need for the developer to be doing administration and provisioning tasks at this point.
If you think you need to set up your own server infrastructure ask yourself one question: is there any specific technical requirement that my application has that can't be fulfilled by existing cloud services? If there isn't, and there probably won't be, you shouldn't be doing ops yourself, especially not in a startup setting where time is absolutely at a premium and you need to be spending all of it on making the best product you can make.
And before everyone tells me that PaaS is more expensive - it's only more expensive if your time is worth nothing. But your time isn't worth nothing - it's probably worth over $100/hr if you are a developer working in the United States. So Heroku ends up not being more expensive at all - especially not before you have to scale.