I put my thoughts together here too: http://swombat.com/2010/12/27/startup-vs-travelling-lifestyl...
With a small, niche, business; I can see it happening no doubt. Like running a small remote consultancy or e-commerce site selling drop shipped products. But a startup needs its founders present, communication face to face is unbeatable and often times your cofounders aren't set to travel the world with you.
So, what's my advice? Live some life while you're building your startup, man; go on short trips, go to Burning Man, visit Prague or fly to the B.V.I for four days. Be sure you have some fun. Once you've either succeeded in making FU money from your startup, or your startup has failed, then go experience the world (you don't need FU money to see the world, hence why you do it even if your startup fails).
My plan? Either FU money, passive income, or a failed startup. Once one of those things happen I'm planning on crewing on a sailboat to get to Europe (or SA, or Aus) and living life for a while outside of the USA (think...years).
I've done my share of remote consulting for venture-backed, aggressively growing startups. For the last 6 years, at least, I haven't physically set foot in any of their offices.
If you want to do it with a whole team, here's how I've done it in the past:
http://www.expatsoftware.com/articles/2006/12/chamonix-branc...
If by startup, you mean a one man website that is some kind of low-maintenance subscription service, then yeah sure, go travel the world.
But if you define startup as I do, as a venture-backed, aggressively growing company employing people, then how the hell are you going to travel the world while managing that? Please tell me if you have a way, because I would be genuinely interested :)