I'm at a loss to see how one could rationally arrive at that conclusion. It's self-evident from the article that McWhorter meant the phrase in the same sense as it's commonly understood. Anything else would be pointless, since he's discussing contemporary usage of the phrase.
One should generally try to figure out what the author's intended meaning was, rather than twist the meaning into something that's easy to argue against but the author would never approve of.
Actually there is: the inclusion of the words "about racism". "Silence is violence" is a catchy slogan. "Silence about racism is violence" is less catchy, so it is not at all implausible that he did not intend it to be read as a slogan, but rather an actual literal assertion.