Disagreements have a key role in building a consensus. But those who disagree with the consensus are usually wrong, because the world is full of capable people with contrarian tendencies and weird ideas.
The consensus might be correct more of the time, but that shouldn’t be a scientific reason to follow the consensus. You should follow the consensus because you have some empirical reason to agree with it and disagree with it if you have reason to believe a better theory.
The only person who needs to defer to an authority reflexively is someone who is completely ignorant of science. They don’t have the sophistication to judge, but thinking that this extends to sophisticated actors is an error. Consensus is for the ignorant.
This is where we disagree. The ignorant believe in their ability to judge. The deeper your expertise gets, the narrower the scope where you trust your judgment becomes. Because you have seen so many ways things can go subtly wrong. And because you have already been confidently wrong so many times.
Science _is_ crowdsourcing the truth, by using ideas that other people generate and then testing them empirically to verify them.
This does not require consensus; rather consensus causes problems which make science worse, disagreements are always better for progress. We don’t have to accept current problems when there are better alternatives (see Lakatos for a good example).