Racism exists everywhere. Living in Japan has been the first place where I've lived as a visible minority and it has been an eye opener. Being obviously different is hard in many ways. There are people who stare at you, or people who wonder if/how you are different from them. There are people who are afraid of you for no reason other than you are different. Very occasionally there are people who are abusive and rude, but most of the time it is mostly a mixture of ignorance and curiosity (and maybe a little bit of fear of the unknown thrown in).
It can be quite a problem for some people. Many people take offence at practically anything and can't let anything slide. They feel it is morally wrong not to take a stand about being treated differently. They feel you are morally wrong if you happily agree to be treated differently.
For me, probably like a lot of people who read HN, I have always been different. I have never fit in anywhere. Even though I was visibly the same as everyone, my way of thinking, my religious beliefs, my values have always been different from the norm. I suffered greatly in countries where everyone assumed that I was the same as them. As a visible minority, I am free to be different. It makes me happy even if sometimes it causes problems.
"Expats in Japan frequently question why I would live in rural Japan with people who don't look [and by extension, think] like me", would be a true statement for me. In my opinion this is likely true of any visible minority population anywhere. It definitely hurts me and I suspect it hurts the author as well, which is probably why he wrote about it in that way.
But the very best explanation is long-form stand-up comedy. If you don't understand, you really should watch this.
This is the most racist thing I read whole day.