The areas of Manhattan you'd be likely to visit are very dense and there are generally no back alleys so, even with the system working as intended, there can be a lot of trash that ends up on the sidewalk waiting to be collected. Yes, in the summer especially, Manhattan gets hot and smelly. There's a reason that a "house in the Hamptons" is something of a meme for wealthy New Yorkers.

When I lived in Manhattan one summer, during a time when the city was "rougher around the edges" than today, I was definitely ready to get out of town on a number of weekends.