Anyone visiting the country but not resident here is also entitled to emergency care, e.g. if you fall off your bike as a tourist and have to go to the hospital, there is no charge. Same if you have a heart attack while visiting.
Am I entitled to NHS treatment when I visit England? http://www.nhs.uk/chq/Pages/1086.aspx
Am I entitled to NHS treatment when I move to England? http://www.nhs.uk/chq/pages/1087.aspx
I wouldn't be too sure about that.
What you're thinking of is the Foreign Earned Income Exclusion, which allows you to exclude the first $95,100 of foreign earned income from your taxes, if you live outside the U.S. more than 330 days a year. In my case there's no real reason to do that, because the foreign tax credit I get for my Danish taxes zeroes out my US tax liability. The main people who benefit from the $95k exclusion are those working in low-tax jurisdictions, e.g. American expats in Saudi Arabia. In their case it allows them not to avoid double-taxation (which is what the tax credit prevents), but any taxation at all.
I'd also spend a good bit of time studying the tax code of any country I start a business in. Some countries, like Australia, tax capital gains at the same rate as income, which is bad news for anyone who put sweat and tears into a startup for years. You'll get punished when you take a nice exit.