As for public safety flights, e.g. police helicopters, they smartly launch from less populated parts of other boroughs, where noise pollution isn't as big of a problem and land is more available. As for medical flights, I don't think there are any that take off from or land in Manhattan. It just doesn't make sense. There aren't helipads near any hospitals that I know of, nor any space to put them, and there's no way you're going to safely land a helicopter on a street to load up a patient. We've already got several world-class hospitals in Manhattan anyway, so a quick ambulance drive is all it takes.
And as for the size of NYC, yes, there's 2 million people who live here in Manhattan full time and then a good deal more than that during the workday. So it's paramount to allocate our scarce resources in a way that maximizes the public good. Spoiling the pleasant enjoyment of so much prime waterfront real estate just for some helicopter flights doesn't make sense and isn't the best allocation of limited resources.
I suspect this is more broad than you intended, but you'd probably find support for it as well.
I guess there are lots of big hospitals outside of NYC proper though.
Look at the bright side: they aren’t Hueys. You’d probably lose your mind if a squadron of angry Hueys thumped up the FDR on the deck. At least the average 206 or Eurocopter you’d see is designed to be somewhat reasonable.