I did this a few years ago.

Grew a site to 10k+ daily uniques while it was running over a DSL connection at my parents house!

"Why not just co-locate a box of your own?" "Co-locate box at Sprocket Data in Dallas for 6 months ($60/mo for 100 GB traffic, 100 mbps burst)"

Because it's too expensive when you're just starting out. I could afford $60/mo, for instance (because I'm older and have lots of kizzash) but I'm hosted for $8/mo with 1.5TB of throughput.

If your site gets much traffic, you'll be over 60GB before you know it. :-o

problem with the $8/mo-type services is that you're limited as to what sort of tech you can use

True, although some providers are more liberal than others. Dreamhost lets you run your own stuff, like ruby, php, rails, etc. But yeah, it's a limited environment, and you're sharing. When you're first starting out, you may not care about that. But eventually ...

The other big downside is that you can't just go in and kill an errant process if your alpha version does something weird.

And you're also in the dark about what those other processes (those not owned by you) might be doing at any given time.

Controlling the entire server as root is just better.

well, to be honest, you get a shell account. I'm hosting stuff on dreamhost, and it's entirely possible to killall my-errant-script when you need to. Just realize that you're a pretty low-priveledge user.

It's a great deal for static file hosting though, if you're a data intensive site as opposed to CPU-intensive site (or even a data-intensive site in addition to cpu-intensive)

If you use FastCGI/Rails with Dreamhost, as soon as you get any traffic they will shut down your account for "Excessive CPU Usage". I've seen it happen to a lot of sites hosted there that are linked from Reddit/Digg.