> No one is sitting on a pile of apartments

They most certainly are. Vast amounts of private real estate in the most prime locations are sitting empty. See the Royal Family's property in Mayfair London for example.