The Georgist answer is that the structures on the property should be assets and that taxes should reduce the land value to $0 and be updated regularly to capture the gains in the land value. That's the theory I ascribe to although I think transitioning from our current society to a Georgist one is incredibly difficult.