Every bylaw and city planning initiative "curtails freedom" for the sake of engineering a better, more efficient community and better quality of life. The issue with (in particular) single family residential zoning isn't that it limits what land owners can do with the land - in the city, this is unavoidable and very much desirable. It's that it artificially incentivizes inefficient, unhealthy and socially disconnected car-centric suburbia.

I'm not sure what it can mean to respect people's property rights with respect to residential-only zoning. A zoning decision is not written into the original deed and can be changed with an exception by the zoning board or a general change in the city/county code. However, the residential-only zoning makes it difficult for one person to use their property as they see fit, for example, building a duplex or rentable apartment onto their house, or opening a small store or tiny school.