Depends on if, by "start small", you mean small effort or small money. Having those things I mentioned shouldn't be expensive, but are huge effort. I want to start small, but if I can't do something as simple as delegate SSH access or search my logs, I can't even test my idea. What I'm talking about is meant to increase the ability to start small for those of us w/out dedicated ops teams.

My point is that you don't need an ops team or lots of ops to being with. A single server can do miracles for a good period of time.

Just use a PaaS like Heroku? Keep the tech simple and vendor locking is minimal.