As long as you make an attempt to respect the culture and (ideally) make some attempt to learn Japanese - I've found only the most rural areas would dare to treat you poorly. They'll treat you well and when you are out of site they'll confide in friends about the "smelly foreign pig". Without a confidant, you'd never be any the wiser. People with tattoos face a higher risk of discrimination than anyone of a foreign (non-Japanese) race in most any part of Japan.

As with any "rules", there are exceptions. But the above has been my experience (month-long travels totaling several years).

Meanwhile, the U.S has no problems yelling out racist slurs or discriminating against others quite blatantly.