This is why property and land taxes are good. Land is, as they say, the only thing they're not making any more of. If you're owning it and not putting it towards any productive use you ought to pay back into the commons for the lost utility.