But nobody ever beat me with a baseball bat. That's a different category of racism that I haven't commented on and that obviously is different from the topic at hand.
As a black person living in America, let me offer my interpretation of GP's statement, then. You can't walk away from racism, because it's always there. It will follow you whether you want it to or not. It's pervasive to the point that even if it's not immediately visible, you can rely on it to surface sooner or later.
Your options are either to exit society altogether or learn to live with it. It doesn't require being beaten with a baseball bat for racism to have a significant impact on your life.
But I can walk away from a situation in which I'm being treated with racism, or a situation in which a narcissist is playing their zero-sum game against me.
I don't hate the narcissist and I wouldn't enjoy their suffering. I just don't want anything to do with them. Nor the racist.
I press life's "restart" button to get a different interaction with a different person, and that has worked well. Even where restarting is massively inconvenient for me. Because ultimately most people don't treat others with racism and most people don't treat others with "evil", so I'm more likely to get a reasonable person on my next attempt.
Obviously if someone is actively striking you or detaining you that's a different category, which is well outside the context of my comment.