Can't speak for him, but I haven't. And I know others that haven't. Doing so only reflects an insecurity in your own opinion. I downvote for ad-hominems, or blatantly (easily verifiable) false claims.

Assuming that others downvote over differing opinions does paint you in the light of doing that yourself, though...