This is a very weak argument that I see too frequently. Simply replace the word "racist" with the word "bigoted". It's no better to be a bigot than to be a racist so the complaint still stands.