Reverse the races and look at how you'd judge a black equivalent. This one "smallest offense" would color your perception of the person permanently.

Should I assume the white person is Italian, Irish, German, Russian, or what?

"Whiteness" is defined in contrast to a person's blackness. You can't talk about racism against whites without first acknowledging a previous racism by whites.