This also happens a lot with Sam Harris, Joe Rogan and anyone who critiques or discusses certain topics. Even though they are obviously not remotely alt-right (indeed they're on the left). It's incredibly frustrating, self-defeating and has a dangerous chilling effect.
Sam Harris has been consistently taking alt-right positions for the past few years (at least). I don't think it's unfair to associate him with the movement.
I think perhaps part of the issue is that lots of Americans who think of themselves as liberal, and who were shocked by the idea of Trump becoming President, actually agree with him on a lot of issues. So, yeah, Sam Harris still presents as an anti-Trump liberal. But if you look at what he's actually saying, he's much closer to Trump than he is to the liberal left. (His atheism is these days just a fig leaf for an anti-Islam stance.)
I had no idea it was so controversial. Every time I've seen coverage of Pinker, his statements are completely taken out of context and he's spun as some sort of alt-right apologist. These are conversations we need to have.
(I first saw him talk about it in his sit-down with Sam Harris.)