They aren't. They also aren't inherently superior and deserving of special privilege by virtue of being older. Otherwise we'd kick off the farmers, the natives who were there before, and every single form of life until the area degenerated to nothing but blue-green algae.
Also, the idea that a position is wrong because it offends or fails to respect someone else's beliefs in arbitrarily selected (but unspecified) ways is one that is offensive and disrespectful.
So there are restrictions on pretty much any use of property in a given town or state. The ability to build a house, run a farm, or build an asphalt plant is allowed or not allowed by the zoning. It's not about having a special "right to farm" any more or less than it's about having a special "right to build a house."
EDIT: Just add that the "right to farm" bylaws are mostly a reminder, especially to newcomers, that people have, um, a right to farm based on existing zoning. So don't move into our country town and then start a petition to close down the dairy farm down the street because it gets a bit smelly now and then during the summer.
That, I suppose, is a long way of saying that zoning can change. And sometimes other regulations change in ways that are inconvenient for some, especially when those people have failed to make a compelling case for why things should not change. We call this "democracy".
You have no legitimate right to demand that anyone do anything with their property based on your preference.
Which is to say you can demand all you like, but until enough people agree with you the right way it doesn't matter.
"Right to farm" rules are an example of democracy in action.
The family of the farmer that I described elsewhere in the thread had literally farmed their 500 acre land for nearly 400 years. Their original land grant from a Dutch Patroon called for annual taxes of 2 pigs and 5 bushels of wheat. They did nothing wrong. They have title to the land.
Why is that family's property rights inferior to those of folks building sprawl homes?