If Warren Buffett wants to increase the value of my land, he can bid up all the surrounding plots, and make bids for my land. Any sane assessment method will see that the value of my land has increased, and will increase my property tax or LVT, and I'll have to pay or face the consequences.
(In the self-assessment case, you can give people the right to refuse to sell, if they are willing to eg back-pay the difference of LVT to the higher price for the last year or so. So people can opt to pay the tax instead of moving out.
To be extra fancy, give the would-be-buyer 1% of the extra tax take to incentivise people hunting for undervalued homes and to compensate for the buyer having had to secure funding.)
Someone who has more of a fear for can put up her self-declared land value. They'll pay a bit more in recurring tax, but would get a significant windfall, if their fear were to come to pass: Yes, there might be some psychological downsides to moving, but getting a extra few million dollars (or whatever) has psychological upsides, too.