Go to any country in the world with an upper-middle class neighbourhood and ask them if it's alright to plop an extra condo or apartment building in the middle.
The only way you'll smoothly get permission is by providing huge, costly, benefits to the local households.
Otherwise it would involve a long drawn-out court battle, or a corruption scheme with the local political potentates, pretty much everywhere where there are such neighbourhoods.
If supply goes up, existing homeowner value goes down, the amount landlords can ask goes down. I don't see any reason it is more complicated than that, the entrenched interest is obvious.
Doubling the housing in a neighborhood could halve peoples entire net worth.
So even if the average home prices slightly increased it would still be the same dynamic.
Increasing housing supply means decreasing homeowner capital and harming the major investment vehicle of a generation. That's politically untenable. Sadly, the pain will be felt by us and our children until we are able to exercise political power, which will require harming our parents retirements. Maybe the answer is renters unions, how many people can the police evict at once?
Young people don't understand their rent is high because they don't vote or exercise political power.