And what was the result of unchecked faulty learning on a corrupt dataset by natural intelligence?

Dark ages where millions died in pointless wars over who's imaginary diety is the true one. A third of europe dying of preventable plague. It took a rigorous system of citation and a publically funded system of education to crawl out of the drakness.

Should we try AI powred dark ages but with resource shortage and nuclear weapons?

Once in a while somebody does some actual on-the-ground research and people are convinced to change their minds through evidence and reasoning. So far AI can't do that, it'll just parrot whatever it hears more often, (apparently) with some degree of randomness.

Maybe it'll learn to adopt a new view, but for now it won't be through actual reasoning.

GPT has had up/downvotes for years. They were used to train ChatGPT. It’s not just “what’s the most likely answer”.

(The most likely answer is the letter “e”.)

It is true but then again, we constantly ajust the trust in sources based on what works and what doesn't.

If I watch a youtube tutorial teaching me how to bake pizza and it comes out totally wrong and a watch another that produces exelent pizza, I disregard the first channel.

And there are many more ways where humans check the reputation of sources automatically.

GPT currently lacks this possibility.