That said I've now moved to a very small town and it is crazy friendly, people will go out of their way to chat to you and be nice (but then again there are plenty of grumpy shop keepers and locals)
So I guess it is the same everywhere - your own little view will be coloured by your own expectations and by how much you are genuinely open to talking to people.
Two weeks before we left we got a dog. We met so many people from the area and the dog was an excuse to start a conversation. Made us feel like we'd been missing out all that time!
It's strange how easy it is to be so alone when there are so many people around. I've also discovered the opposite to be true, and the principle also carries over to large and small schools and large and small companies. When there are few people, they are more relational, but when it's impractical to greet everyone you meet, you may greet no one.
First of all, I've seen Japanese (who are strangers) strike up conversations on occasion in Tokyo: at the coin laundry, at the bakery, on elevators, and so on. It's not like a social taboo or something. But yes, in general, people keep to themselves.
That being said, in my experience, people in New York and Los Angeles also keep to themselves. My instinct is that the apparently introverted nature of people in Tokyo has nothing to do with Japanese culture and everything to do with the fact that it is a big city in a developed country. Perhaps someone from London can bring in another data point?
If my experiences in rural Japan are at all normal, Japanese in rural Japan are at least as talkative as those in the Southern US (where I'm from).