Censorship is non-consensual filtering of information (the mediator censors without consent of producer and consumer of information).
This does not mean we can't control the spread of disinformation at all: by investing in education, and educating people about the importance of verification in general or formal verification in specific where possible, society could invest in infrastructure that helps in weeding out false information without twisting the consumers hand willy-nilly.
As an example: without any need for censorship, if someone claims to give a proof for a theorem that supposedly follows from set.mm I can consent to censorship by voluntarily using MetaMath to verify it without even trying to understand the proof myself, and then discover that the proof is false, without some intermediary of information needing to censor anything for me.
This is kind of the underlying theme behind the "blockchain craze", educating people the surprising applicability of logic/mathematics to the real world, even if it is very hard to design and prove the security of such systems, and to interpret the assumptions of such designs with desirable properties as being the requirements of infrastructure we need to build in order to enjoy such a decentralized commons.
The big rub in your belief is that social groups don't scale. In a group of size of 100 people or so, standard social techniques (such as peer pressure and ostracization) can keep order. But at 10,000 people, that doesn't work. Delegation does scale. Society and civilization essentially works by creating structures that can impose norms, rules, and punishment among others and offloading the cost of doing so to people who specialize in those tasks. Now, that doesn't mean that there aren't issues of "who watches the watchmen," but it does mean that that is the question you should be asking.
One example of the problems of decentralization is the modern information delivery mechanism. Because it's feasible to serve smaller groups, our social groups have broken up into smaller echo chambers where we're fed only the information we want to hear. The ease of spreading information is also ease of spreading misinformation, and objection to censorship means there's no way to control spread of misinformation.