It depends on where you live. There are areas where you never hear of it, and others where being carded once or twice a year is the norm.

It doesn't happen that often but it does happen. Foreign residents are required to carry a resident card at all times and the police are allowed to check for it. This can come with a search.

Japanese nationals don't need to carry any form of ID by law, so you can see that how the police choose who to stop might be somewhat problematic.

I lived in Japan for over 3 years and I was never stopped. It can happen, technically, but I don't think it happens all that often.

Personally I have never experienced a random stop here in Japan. None of my friends here have worried about such either.