> Anyway, the builders all say there is no financial incentive for them to build affordable housing, they make so much more on luxury buildings.

This is only really a problem insofar as building luxury homes reduces the throughput of new supply because they take longer to build than affordable housing. In terms of overall market effect, as long as you're not allowing places to sit empty, it doesn't particularly matter if you're adding new homes at the top or the bottom of the market.