affordable housing is a pretty loaded phrase at this point. In fact it has come to just mean government-subsidized housing. So by that definition you are probably correct, people aren't often moving from housing projects to brand new condos. Nobody is saying that they do.

luxury condos is also a loaded phrase. In an expensive city, all housing is expensive, even if it is 50 years old and used to house working class people. So-called "luxury housing" is just regular-sized new apartments, with a few thousand dollars of added perks like sleek recessed lighting and upgraded appliances to help sell it.

If you take out those loaded phrases, people are absolutely moving from existing, mid-tier market rate housing into only slighty more expensive new construction market rate housing. The more new construction housing you build, the more supply you have, and the more prices come down. Which allows market-rate housing to become affordable to more of the population. That was the case 10-15 years ago, and could be true again in the future if we just let people build enough housing to match the incoming demand of new residents moving into cities.