A decentralized system is not controllable directly, but can only be as smart as the mean of the population[0]. Also, it also has a mechanism of control - propaganda. It's not all that effective, because you can have competing groups countering one another, but the side effects of that are relevant here: people take sides and reason by soundbites, while facts and critical thought get crowded out. Which means the more competing propagandists you have, the dumber the system becomes, even if all propaganda attempts cancel each other out.
So, on one hand you have a system with a very good quality ceiling but easy to manipulate; on the other hand, you have a system more resistant to direct manipulation, but consequently with a much lower ceiling that that also goes monotonically down as attempts to manipulate the system increase. Or, in other words: good results with high risk of bad results, vs. consistently shitty results.
Can't we do better? Are we doomed to choose between only those two?
--
[0] - Assuming a normal distribution, by the central limit theorem. If that doesn't hold, things could get even worse.
They aren't guaranteed to be smart or just (sadly there probably isn't anything which qualifies) but they are are stable in a "not likely to piss off everyone into an insurrection" way.
No. It wouldn’t. When that kind of power (arbiter of truth) is placed in a single point, that point will become the highly contested point of control. Eventually, inevitably, the body would end up turned to some ruthless power of the day, and would serve as the most phenomenal propaganda machine.
“The Ministry of Truth is pleased to announce that bootlace production is up a thousandfold this quarter, due to the heroic efforts of the King. You may now cheer.”
Total decentralisation is the only sane route - you build a bulwark out of billions of human minds, and you build inertia into your system such that emotional volatility led by events or propaganda efforts is smoothed out.
This is basically the concept of democracy and electoral cycles - group consensus on what reality is and how it should be responded to, with a temporal buffer (the election cycle length) to smooth out rash decisions.
Our systems need amendment to better smooth our inherent volatility, which has been provoked by our exponentially growing sphere of information - but this can’t come at the cost of handing control over something as fundamental as the idea of truth to some technocratic committee. That’s how you end up with gulags.
Edit/addendum:
Further musing on temporal smoothing of volatile and labile humanity led me to recall the idea of Concents in Anathem - groups are sequestered away for a year, a decade, a century, or a millennium, and are allowed only at those intervals to interact with the outside world. This monkish dedication to record preservation minimises informatic drift within the group, and provides a continual negative feedback factor to the unstable curve of truth, thus preventing the rewrite of the past, and therefore the present and future.