The more difficult question, I think, is: why should an average Joe trust the (presumably government or corporate) AI over their favourite worldview-confirming sources?

Truth is not an inherent reality of the universe. There is matter and energy in various juxtapositions, but no 'truth', except if by truth we mean correctly describing a physical order of things. But that's hardly the interesting part or what we usually want to know when we speak of truth.

At best, truth in things like politics and such requires a definition first, and so the problem of what's true is just moved to "according to whose definition?". Is "wealthfare harmful", for example, is not about knowing or not knowing its cost, but more about whether you think the costs are justified. One can legitimately believe that poor people should be left to rot if they can't afford healthcare, for example.

Second, why would an "advanced AI" stick to the truth any more than a human would? Especially if distorting it brings large benefits for it?

When I ask someone if "welfare is harmful", I ask for their opinion on the relative impact it has on various issues we want to optimize for, which mostly boil down to short and long-term happiness of people. It's a difficult question, but still a factual one. The list of things we want to optimize for is finite, there is some ordering to it, and the impact of various actions on those things is, in principle, calculable.

Even the question of "can welfare be justified" is an objective one. There is some model for "justified" that you have, and the data either lets you justify welfare or does not (and tells you the degree to which it is so, if your thinking process is worth anything).

It's only when people start pushing their stupid agendas and start putting their personal interests in focus that the whole thing gets "difficult".

--

Either way, I only said that a sufficiently smart AI could know the truth (or at least a good approximation of it). I didn't say it will necessarily tell it to people who ask. But an AI that can't correctly from data would be a pretty shitty AI. It's enough that human brains have this problem.

I think you might be surprised at the diversity of answers you get for this, particularly if you actually get people's unfiltered opinions and not what they think you want to hear. Try it, sometime - ask 20 random people (not connected to yourself) what their biggest problem is and what they most care about in their life. You'll get some commonalities (earning a living is a big one, as is social acceptance & validation), but as soon as you start digging into how do you wish to earn a living and what do you consider an ideal world, it very rapidly diverges.

"It's only when people start pushing their stupid agendas and start putting their personal interests in focus that the whole thing gets "difficult"."

That's called "being human". What would you rather they do, defer to your agenda?

I don't think it's calculable, even in principle. Except in trivial issues, people can (and do) disagree on very fundamental ideas about what we want to optimize for and which direction we should optimize them.

>It's only when people start pushing their stupid agendas and start putting their personal interests in focus that the whole thing gets "difficult".

Why wouldn't the people put their "personal interests in focus"?

Whose interests they should put to focus? And says who?

Your mistake is assuming the data is objective in the first place, and doesn't, a) contain the societal bias at the time it was written, b) contain the bias of the authors that made the study, and c) contain any inherent bias in the metrics used.

All the AI knows is the training set. This is given a necessary bias by the researchers in the process of creating the AI. The best an AI can do is exaggerate the biases that it learned from the training set (Which is, after all, the main point of using AIs -- to exaggerate favorable biases to produce data that the researches believe is valid).

[1]: www.badscience.net/2009/10/and-now-nerd-news/ [2]: https://www.researchgate.net/topic/Negative-Data-in-Science [3]: http://www.nature.com/news/half-of-us-clinical-trials-go-unp...

The rise of far-right politicians in Europe indicates that America is not alone here.

External reality might. Reality of what people want, what we think is best, etc, doesn't.

And even the external reality part is close to impossible to access and convey to others whether it "adds to a coherent whole" in all situations or not.

I think we're talking about different things: the current state of AI vs (what's currently) science fiction.

Half of my response still applies though. The thing with trust is that you can always call it into question. Give people a powerful AI-powered fact-finding engine, and there will still be those questioning whether you could trust the tool.

This isn't a greenfield.

There's been some research on this. I think I even saw at least one article about it here on HN, though I can't find a link any more.

What I remember seeing is a kind of world (?) map of the Internet with a set of different-colored interconnecting webs of sites.

There is one set that deals mostly in verifiable facts, usually based on genuine, usually academic research. Universities are major nodes in this net, but you get some overlap with "mainstream" news sources, Wikipedia and so on.

There's another major set that includes InfoWars, Collective Evolution, Natural News, Deepak Chopra, Mercola and other Anti-Vaxxers, various AGW deniers, and other well-known sources of bullshit.

Both of those networks have many more connections within themselves than to the other clouds, and have their own frequent-viewer audiences who will frequent the one set more than the other.

There were, as I remember, other sets that were less significant.

Purely on probability, a story will be dramatically more likely to be true if originated and/or propagated in the first of those clouds than the second.

Mapping these webs is a pretty mundane mechanical task, yet it can provide an excellent filter for some classes of information.

Other stuff is "squishier" because it's more opinion than fact. Politics and policies will be found here, and you're likely to see (I don't remember if the article I have in mind showed this) "conservative" vs. "liberal" networks rather than "fact" vs. "bullshit". On the other hand, there are experts on many aspects of politics, and if you're looking for ideas on policy that are informed by history and research, you're more likely to get them from professors of history, economics and law than from Fox News or televangelist channels.

I suspect there are a lot more inferences that can be drawn, via probability or Bayes, from the Internet sources of information, its propagation behavior, and the shape of the networks it travels.

Similarly, there's a wealth of information to be gained from analyzing the language of an individual page. Emotional words vs. dispassionate ones; weasel words like "probably" or "it was said" as opposed to "according to" or links to other sources. Bullshit is often simplified for consumption by simple people, while academics often carelessly lapse into "high-bred" or technical language.

Some of those features could be gamed, simulated, faked, and certainly will be. Many bullshit articles in the health, field, for instance, now link to "outlier" or discredited papers in PubMed to enhance their credibility. But like our grandparent poster, I think AI, working patiently and dispassionately, will be able to continue to find hints to distinguish the wheat from the chaff.