Urban noise is like a noisy clock ticking in the other room, or an air-conditioning system blowing air around. 99% of the time you honestly have absolutely no perception of it, but that last 1% of the time you can't understand how it wasn't driving you crazy before.
And I'm a little bemused how so many people can cope with living in urban environments rather than rural ones. The noise, smells, neighbors right through the wall, traffic, etc. But I suspect the majority of HNers live in urban environments. Noise tolerance, like urban, population density and crowd tolerance, is personal and subjective and it's tricky to rationalize why people do or do not want to "control" it.
I fall asleep more easily if I have chatter around me (so I listen to podcasts). This is unusual but not something I need to rationalize away. It's just how I tick and other people tick other ways.