San Francisco's approach to housing is so endlessly entertaining. They want to allow developers to build higher-density dwellings but force them to do so at a loss. Gee, I wonder how many developers will voluntarily lose money.

The next logical step is to subsidize the building of the units made unaffordable by regulations, and this is how you get politically funded housing and the patronage machine keeps growing.

Is it possible they don't really want that higher density development and create it that way on purpose to give the illusion?

You just nailed it. That it is the virtue signaling politics of San Francisco. The policymakers in California are only interested in exploiting politics to posture for purposes of self-esteem, branding, status and tribalism.

It reminds me of Marc Andreesen who penned the article "IT’S TIME TO BUILD" while living in the most anti-building neighborhood in the whole Bay area, full of billionaire's called Atherton.

https://future.a16z.com/podcasts/its-time-to-build/