Surely context is everything here, no? It’s not the topology of a Swastika that is somehow racist, using a Swastika as a link to the Nazis is. This is generally unambiguous too, in my experience.

I would say a good first-pass filter should apply a strong, but perhaps not overwhelming prior probability to the Swastika being a racist symbol.