I agree with you about housing but natural rights exist regardless of legal recognition. Rape, murder, genocide, and slavery are all wrong regardless what a bunch of people agree about. That's what natural rights are. They are things that can be violated by an all powerful dictator. But the dictator would be wrong and terrible for doing so.

That being said, I'm hard pressed to call desired housing density a natural right.