People have been socialized to believe in the American Dream(tm). A detached single family house with a white picket fence and 2.3 kids is how the world ought to work. Allowing a duplex or (gasp) a 6 unit apartment building to be permitted would irreparably damage the character of the neighborhood.

Even with government intervention in the form of Fannie Mae and Freddie Mac underwriting 30 year mortgages, toll free highways to link suburbs with job centers, and mandated parking at every building, single family homes are not affordable.