Trump was viewed as problematic before November 2020

Yes, specifically when he refused to commit to accept the results of the election if he didn’t win. As the events leading to and including Jan 6 showed, this was not just bluster, he used all means including illegal ones to try to hold on to power after losing the election. That is what made him the most problematic, someone who doesn’t want to play by the rules of democracy.

That was an example so straightforward that it's hard to imagine any honest person looking at it and saying "Yeah, that's all right".

The people who opposed him before could cite numerous other examples, and to be honest I can't really imagine how many people changed their minds over that one if they'd accepted everything else. If they do exist, it's hard to imagine that it came as a complete surprise. I would imagine that in retrospect they'd have to say, "Yeah, maybe the people whose guy he wasn't weren't completely off base".

That is, it's not just the usual "not our guy" thing. This was different in kind, not just in amount. Maybe some people opposed him just because he wasn't their guy, but there were things that should have broken the mold of "he's our guy therefore he's ok".