This is called the Halo effect: someone who is good in one way is surely good in all ways.
If Terence Tao posted on his blog tomorrow that he has video proof that Hillary Clinton is a secret lizard person, I would certainly be shocked, and I might even read what he has to say, but I would still immediately classify him as a nutty conspiracy theorist (one who espouses a nutty conspiracy theory). It wouldn't really diminish everything else he's done, and I would view it as an incredibly sad case of an otherwise amazing brain getting infected by a particularly virulent mind-virus.
Say Joe Rogan split his segment into two halves, and the first half was devoted to someone arguing that the 14th century was invented by the Illuminati in order to hide the fact that Finland doesn't really exist and is just a cover to hide the fact that Russian fishermen discovered aliens there, who took their bodies and are now infiltrating society, etc. etc. The second half was devoted to an actual historian saying "um, no, that's really ridiculous for many reasons". Would you call that fair and balanced? Is that a good point-counterpoint? Hell no. It's the classic false equivalency, fake-centrist B.S. that is used so often to promote crazy and dangerous ideas.
The truth is subtle, nuanced, and complex. It takes long, in-depth study to understand. It's not always obvious to a layperson who is "debunking" whom. Just as we don't automatically expect a beautiful or useful crop to outcompete noxious weeds just because we like them better, we should not expect the truth to outcompete crazy in 20-minute podcast segments. Crazy is much better at spreading into brains than the truth, because that's exactly the selection pressure that led the crazy to evolve in the first place. True ideas don't evolve by being extremely good at sticking in your brain; they evolve when humans sit down and realize that that's not the appropriate arena for them.
Did you watch the posted ten minute video? Have you watched any of his podcasts in full? He explains why he has those guests on and concedes with some gratitude the idea that he ought to have more guests on that can communicate the opposing views, especially soon after a controversial guest.
The alleged harm he’s perpetrating in having conversations is way less dangerous than a full-steam-ahead into uncritical, contrived, and corrupt thought control from established institutions and governments.