In the end I'm a big fan of owning the hardware and dealing with the issues yourself.
That said, a small ops team (three people) can manage thousands of machines without much difficulty, and you can probably share two of those with the dev team as Dev-Ops guys, with one grizzled old sysadmin to handle the nastier problems.
I think that a lot of companies shy away from buying hardware for two reasons. One, because they haven't sat down and looked at the long-term ROI, and two, because for the past five years or so, every piece of IT media has talked about how stupid buying hardware is, and how we're all just going to go live in the cloud.
Sounds quite a bit like the NoSQL rhetoric, actually.
The reality is that at a certain scale, it's cheaper from a strategic and/or monetary standpoint to have in-house hardware.
Don't get me wrong -- there's definitely a big home in the ecosystem for cloud hosting. Probably less than ten percent of businesses need to buy their own hardware. But the ones that do, need dedicated hardware in a bad way.
I worked for a company that contracted with a guy whose entire business was based on this model. He was the hardcore sysadmin and would juggle tons of servers and administrative config for you. He trained the devs to handle all of the mundane ops stuff, with him as backup.
Really great system. It was really empowering for us devs, too.
The real killer with the cloud is bandwidth and storage. With my colo deal I am paying about 5-10% of what bandwidth would cost at Amazon. Not to mention I can put any kind of storage I like with the actual cost per Gb to buy outright being equal to monthly cost on EC2. Go figure!
Cloud has its place to handle spikes of demand, but it is really not cost effective to scale.
There are advantages of a Hosting company buying hardware for you and amortizing it. It is impossible for a small startup buying < 5-10 servers to achieve say a cost of Rs. 100-200 USD per month per server including amortization ( say 24 months ), bandwidth, power and remote hands per server. And I super agree with the fact that it is easy to run high bill on cloud hosting where you pay through your nose for everything except perhaps the base level machines for the privilege of being able to burst your load to 100x. Only about 2% of the customers on cloud actually require burst capacity beyond 2x. It is easy enough to achieve cost savings of 30% with always-on 2x capacity using standard dedicated servers from a dedicated server focused hosting company than buying capacity on the cloud and using maybe burst capacity 5% of the time.
It is expensive, but not as expensive as something like heroku or aws for equivalent cores/RAM, and you get real hard drives.
In certain datacenters (think might be dallas05?) you can even get a 10Gbps uplink brought to a box so they have pretty big pipes available.
In Softlayer's Dallas facility (selected when ordering a server) 10 Gbps is an option.
The maximum amount of bandwidth available is 20TB.
Sometimes you can get more options by contacting their sales people directly instead of using the shopping cart.
When you're starting out, you go with something on the cloud -- EC2, Rackspace, Softlayer, Linode, or whatever cheeses your hamburger.
That's just common sense. There's no reason for a fresh web or mobile startup to have tens of thousands of dollars of server inventory, bought for the anticipated rush of people that want to get in on the 'social synchronized tuba-thumping' craze.
But when you hit the point where you're spending $10k a month on cloud hosting and your team is having to put out performance fires thanks to the shared hardware, why not just buy servers and throw them in a datacenter?
Amortized over a year, you'll spend a lot less on capital (network and hardware), and a lot more on labor, as you'll need at least one more team member to handle the on-call rotation load.
Apologies if this is rambling -- it's about 7:30am here, and I'm a bit low on sleep. :)