If that's all that's keeping you back, you should check if the city/state you wish to live in has what's usually called a homestead exemption. Many places will reduce property tax if you meet a few criteria. The criteria varies, but it usually revolves around being on a fixed income and living on the property.

Florida has property tax, but they don't have an estate tax, which probably influences people to retire there...

For information your property tax (council tax) for the most expensive house in Central London can be less than an average 4 bed house in a mid ranked US state.