No my friend, it's also insulting via the "why you picked Karen (implicitly white woman) as a reference for an insult" route. You may use the insult on anyone else but the fact that you picked "white woman" as the reference means you believe that this is a defining enough trait to associate with them, you consider white women the embodiment of that negative trait and as representative of it.

It's the same reason calling someone a "fag" is meant as an insult for both that person (even if they are straight), and for homosexuals since you obviously consider this trait of theirs only suitable for an insult. It's an attack both on the people you address it to, and on the ones you based your insult on. Something an uneducated bigot would readily use but never realize why. They usually feel threatened by everything they don't understand or like (different gender, color, sexual or religious orientations) and "weaponize" that as a cover for their own ignorance. ;)