If the baseline is a duplex, where I'm already depending on how good the sound insulation is, then I say go ahead and make it 20 stories.

What's so hellish about a large building?

If the sound proofing is essentially 100% then I'm fine with apartments, but every apartment I've lived in you could hear people noises all around and it's annoying and sometimes anxiety inducing when you're trying to get some sleep or concentrate on a task, hence I live in the burbs where I might hear the occassional loud car go by but never people stomping, fighting, or f..king