I would far rather restrict their right to vote than their right to speak. If they're sufficiently influential, that also means that everyone's vote who doesn't fall for their charm is worth just that much more.
I'm very opposed to limiting either, the US is already more restrictive with regards to both voting rights and speech than I'm comfortable with. I'd be very worried about gating voter rights to some kind of checklist of thought, removing Coting access from those deemed to be unworthy of casting a vote based on what they think or say.
Interesting point of view/source of internal tension there.
There are historical examples of why trusting the mob goes poorly, but keeping that on the table fundamentally ruins the point of a democratic society. Its all well and good to say we need an educated populace, but the system should offer the public a chance to live up to the ideal rather than guard against their failure.
With that said, what's the problem with people having ideas that seem crazy and baseless? Does it really harm you at all if someone else believes 5G causes covid? And does it harm you enough to say that their speech should be limited rather than free?