I think in that case, it would probably be a problem in most countries (unless, say, you're from the EU and pick another EU nation etc). There are nations like Costa Rica that have an entrepreneur visa, but AFAIK you have to employ certain number of locals and show some upfront investment to get it.

If I were in the OP's position, I believe I'd incorporate (if I really need to at all) where I have citizenship, then work on the product wherever I choose.

If you're talking an office with employees etc, that's a different story, but if it's a couple ppl working from an apt or cafe, perhaps with other staff telecommuting from where they live, I can't imagine this would a problem anywhere.