I could see replacing "single family" with some sort of density restriction (people per acre?). What a home entails is not simply a dwelling + piece of land to its borders, but the community in which it belongs. Communities have generally decided that a predictable nature here is desirable, and thus have enacted zoning restrictions.
If you look at Houston, the largest city with no zoning laws, the private sector has effectively re-implemented zoning via contract law in huge sections of the city, in the form of subdivisions where the initial developer set up a homeowner's association, empowered via deed restrictions, that implement similar rules about not only density, but even things like what color you can paint your windows. People clearly seem to like living in that kind of community-managed neighborhood. At least doing it via the actual government, rather than these weird quasi-governments, is more transparent and democratic.