I would expect this isn't the on-off switch they conceptualized, but I don't know enough about how different LLM providers handle news search and retrieval to say for sure.
No... the problem is that it cites Wikipedia articles that don't exist.
> ChatGPT linked to a non-existent Wikipedia article on the “European Union Enlargement Goals for 2040”. In fact, there is no official EU policy under that name. The response hallucinates a URL but also, indirectly, an EU goal and policy.
https://en.wikipedia.org/wiki/Wikipedia:Articles_for_deletio...
A recent Kurzgesagt goes into the dangers of this, and they found the same thing happening with a concrete example: They were researching a topic, tried using LLMs, found they weren't accurate enough and hallucinated, so they continued doing things the manual way. Then some weeks/months later, they noticed a bunch of YouTube videos that had the very hallucinations they were avoiding, and now their own AI assistants started to use those as sources.
I have seen a few cases before of "hallucinations" that turned out to be things that did exist, but no longer do.
It's not bad when they use the Internet at generation time to verify the output.
Pre prompting to cite sources is obviously a better way of going about things.
IDK how you people go through that experience more than a handful of times before you get pissed off and stop using these tools. I've wasted so much time because of believable lies from these bots.
Sorry, not even lies, just bullshit. The model has no conception of truth so it can't even lie. Just outputs bullshit that happens to be true sometimes.
That seems to be the real challenge with AI for this use case. It has no real critical thinking skills, so it's not really competent to choose reliable sources. So instead we're lowering the bar to just asking that the sources actually exist. I really hate that. We shouldn't be lowering intellectual standards to meet AI where it's at. These intellectual standards are important and hard-won, and we need to be demanding that AI be the one to rise to meet them.
These grifters simply were not attracted to these gigs in these quantities prior to AI, but now the market incentives have changed. Should we "blame" the technology for its abuse? I think AI is incredible, but market endorsement is different from intellectual admiration.
For example, having a single central arbiter of source bias is inescapably the most biased thing you could possibly do. Bias has to be defined within an intellectual paradigm. So you'd have to choose a paradigm to use for that bias evaluation, and de facto declare it to be the one true paradigm for this purpose. But intellectual paradigms are inherently subjective, so doing that is pretty much the most intellectually biased thing you can possibly do.
I've seen a certain sensationalist news source write a story that went like this.
Site A: Bad thing is happening, cite: article Site B
* follow the source *
Site B: Bad thing is happening, cite different article on Site A
* follow the source *
Site A: Bad thing is happening, no citation.
I fear that's the current state of a large news bubble that many people subscribe to. And when these sensationalist stories start circulating there's a natural human tendency to exaggerate.
I don't think AI has any sort of real good defense to this sort of thing. 1 level of citation is already hard enough. Recognizing that it is citing the same source is hard enough.
There was another example from the Kagi news stuff which exemplified this. A whole article written which made 3 citations that were ultimately spawned from the same new briefing published by different outlets.
I've even seen an example of a national political leader who fell for the same sort of sensationalization. One who should have known better. They repeated what was later found to be a lie by a well-known liar but added that "I've seen the photos in a classified debriefing". IDK that it was necessarily even malicious, I think people are just really bad at separating credible from uncredible information and that it ultimately blends together as one thing (certainly doesn't help with ancient politicians).
Also, is attributing, without any citation, ChatGPT's preference for Wikipedia to a reprisal to an active lawsuit a significant issue? Or do the authors get off scot-free because they caged it in "we don't know, but maybe it's the case"?
And the worst part about the people unironically thinking they can use it for "research" is, that it essentially supercharges confirmation bias.
The inefficient sidequests you do while researching is generally what actually gives you the ability to really reason about a topic.
If you instead just laser focus on the tidbits you prompted with... Well, your opinion is a lot less grounded.
>...it finally told me that it generated « the most probable » urls for the topic in question based on the ones he knows exists.
smrq is asking why you would believe that explanation. The LLM doesn't necessarily know why it's doing what it's doing, so that could be another hallucination.
Your answer:
> ...I wanted to know if it was old link that broke or changed but no apparently
Leads me to believe that you misunderstood smrq's question.
A critical human reader can go as deep as they like in examining claims there: can look at the source listed for a claim, can often click through to read the claim in the source, can examine the talk page and article history, can search through the research literature trying to figure out where the claim came from or how it mutated in passing from source to source, etc. But an AI "reader" is a predictive statistical model, not a critical consumer of information.
Yep.
Including, if not especially, the ones actively worked on by the most active contributors.
The process for vetting sources (both in terms of suitability for a particular article, and general "reliable sources" status) is also seriously problematic. Especially when it comes to any topic which fundamentally relates to the reliability of journalism and the media in general.
I have one example that I check periodically just to see if anybody else has noticed. I've been checking it for several years and it's still there; the SDI page claims that Brilliant Pebbles was designed to use "watermelon sized" tungsten projectiles. This is completely made up; whoever wrote it up was probably confusing "rods from god" proposals that commonly use tungsten and synthesizing that confusion with "pebbles". The sentence is cited but the sources don't back it up. It's been up like this for years. This error has been repeated on many websites now, all post-dating the change on wikipedia.
If you're reading this and are the sort to edit wikipedia.. Don't fix it. That would be cheating.
Imagine if this was the ethos regarding open source software projects. Imaging Microsoft saying 20 years ago, "Linux has this and that bug, but you're not allowed to go fix it because that detracts from our criticism of open source." (Actually, I wouldn't be surprised if Microsoft or similar detractors literally said this.)
Of course Wikipedia has wrong information. Most open source software projects, even the best, have buggy, shite code. But these things are better understood not as products, but as processes, and in many (but not all) contexts the product at any point in time has generally proven, in a broad sense, to outperform their cathedral alternatives. But the process breaks down when pervasive cynicism and nihilism reduce the number of well-intentioned people who positively engage and contribute, rather than complain from the sidelines. Then we land right back to square 0. And maybe you're too young to remember what the world was like at square 0, but it sucked in terms of knowledge accessibility, notwithstanding the small number of outstanding resources--but which were often inaccessible because of cost or other barriers.
Not to mention, the AI companies have been extremely abusive to the rest of the internet so they are often blocked from accessing various web sites, so it's not like they're going to be able to access legitimate information anyways.
One way to successfully use LLMs is to do the initial research legwork. Run the 40 Google searches and follow links. Evaluate sources according to some criteria. Summarize. And then give the human a list of links to follow.
You quickly learn to see patterns. Sonnet will happily give a genuinely useful rule of thumb, phrasing it like it's widely accepted. But the source will turn out to be "one guy on a forum."
There are other tricks that work well. Have the LLM write an initial overview with sources. Tell it strictly limit itself to information in the sources, etc. Then hand the report off to a fresh LLM and tell it to carefully check each citation in the report, removing unsourced information. Then have the human review the output, following links.
None of this will get you guaranteed truth. But if you know what you're doing, it can often give you a better starting point than Wikipedia or anything on the first two pages of Google Search results. Accurate information is genuinely hard to get, and it always has been.
Still not enough as I find the LLM will not summarize all the relevant facts, sometimes leaving out the most salient ones. Maybe you'll get a summary of some facts, maybe the ones you explicitly ask for, but you'll be left wondering if the LLM is leaving out important information.
Right. Let's talk about statistics for a bit. Or let's put it differently: they found in their report that 45% of the answers for 30 questions they have "developed" had a significant issue, e.g. inexisting reference
I'll give you 30 questions out of my sleeve where 95% of the answers will not have any significant issue.
Neither is my bucket of 30 questions statistcally significant but it goes to say that I can disprove their hypothesis just by giving them my sample.
I think that the report is being disingenious and I don't understand for what reasons. it's funny that they say "misrepresent" when that's exactly what they are doing.
Disclaimer: Started my career in onine journalism/aggregation. Hada 4 week internship with the dpa online daughter some 16 years ago.
Imo at least
There’s no such thing as unbiased.
The BBC is the broadcast wing of the Guardian.
Perhaps the real bias was inside us the whole time.
The Guardian is at best centre-right.
Next you’ll try to convince me that Starmer’s Labour is left wing or the Lock Ness monster is real.
However, that is currently not reflected in electoral politics or the media. The farthest left are currently the Greens, at best centre-left. On the right and far right there are Tories and Reform.
With this in mind, 45% doesn't seem so bad anymore
Probably the most impactful "easy A" class I had in college.
I was on my highschool's radio station, part of the broadcast media curriculum. It was awesome.
That early experience erased any esteem I had for mass media. (Much as I loved the actual work.)
We got to visit local stations, job shadow, produce content for public access cable, make commercials, etc. Meet and interview adults.
We also talked with former students, who managed to break into the industry.
Since it was a voc tech program, there was no mention of McLuhan, Chomsky, Postman, or any media criticism of any kind.
I learned that stuff much later. Yet somehow I was able to intuit the rotten core of our media hellscape.
Maybe we complained with enough concrete examples of how absolute shit editors and summarizers are now.
POSIWID
Much of this is probably solvable, but it very much not solved.
> 45% of responses contained at least one meaningful error. Sourcing [...] is 31%, followed by accuracy 20%
And you can see the reason they think this is important on the second page just after the summary.
> More than 1 in 3 (35%) of UK adults instinctively agree the news source should be held responsible for errors in AI-generated news
So of course the BBC cares that Googles summary said that the BBC cites pornhub when talking about domestic abuse (when they didn't), because a large portion of people blame them for the fact that a significant amount of AI generated crap is wrong.
Other issues: the report doesn't even say which particular models it's querying [ETA: discovered they do list this in an appendix], aside from saying it's the consumer tier. And it leaves off Anthropic (in my experience, by far the best at this type of task), favoring Perplexity and (perplexingly) Copilot. The article also intermingles claims from the recent report and the one on research conducted a year ago, leaving out critical context that... things have changed.
This article contains significant issues.