They won't mind at all if they never intend to do X and it helps them get a discount.

Poison pills like that are common in conservation easements, for example. Our family owns land with an easement which, if I recall correctly, states that anyone wishing to break the easement will owe back-taxes reaching back to 1992, with compounded interest, on the development rights to the land.

In return, we were able to keep the family farm when my grandparents passed away.