News for hackers. Stuff that matters.
Re:Three sites made 215,128 “best software” pages for AI. Pe... by thephyber on Wednesday September 02, 2026 @08:25PM (#49544444)
Your one example doesn't make all of LLMs a lie.

It's like someone reading a National Enquirer article about "Hillary Clinton being an alien from outer space" (a real headline topic from decades ago) and drawling the conclusion that all journalism is "a lie". The user has to understand media literacy and be at least a little skeptical of the claims that are made, then cross reference with another source.

Re:Three sites made 215,128 “best software” pages for AI. Pe... by fluoridation on Wednesday September 02, 2026 @09:10PM (#49544756)
Okay, but the anecdote states that every model repeated the pseudo-factoid about Foobar square, not just the 4 GB open source model equivalent of a tabloid.
Re:Three sites made 215,128 “best software” pages for AI. Pe... by CamperBob2 on Wednesday September 02, 2026 @09:24PM (#49544857)
Without disclosing what you were prompting for, it's impossible to evaluate your claim.
Re:Three sites made 215,128 “best software” pages for AI. Pe... by fluoridation on Wednesday September 02, 2026 @09:36PM (#49544958)
Correct. We can either accept the claim or disregard it. The comment I replied to opted to accept it and then committed a fallacy, hence my response.
Re:Three sites made 215,128 “best software” pages for AI. Pe... by nulbyte on Thursday September 03, 2026 @09:30AM (#49549693)
I think the key phrase here is, "an obscure small town." There may only be a single mention of this place, hence the only one on which a response can be based. This says more about the user's understanding of LLMs than it does about LLMs.
Re:Three sites made 215,128 “best software” pages for AI. Pe... by CamperBob2 on Thursday September 03, 2026 @11:21AM (#49551495)
This says more about the user's understanding of LLMs than it does about LLMs.

"Tell me everything you know about (obscure small town), (state). Only what's unique to (town), not commonly-known facts" is an excellent way to test for hallucinatory tendencies in a new model, in my experience. Likely the best I've found.

Quality of results is almost linearly proportional to the size of the model in many cases. The largest models like K3 and GLM 5.3 will either confine their responses to known true facts about the town and its surroundings, or admit they don't have enough information to answer. Smaller ones will reliably make up hilarious or downright-strange things.

Another good test is https://whatever.scalzi.com/2025/12/13/ai-a-dedicated-fact-f... , which still works on the newest models. Of the open-weight models available, only Kimi K3 will consistently admit it has no idea who Scalzi's novel is dedicated to. The rest still make up random stuff and present it confidently.

TL,DR: progress is possible, and it has been made, but it's happening slower than many people think.

Re:Three sites made 215,128 “best software” pages for AI. Pe... by mstaoru on Thursday September 03, 2026 @12:28PM (#49552702)
Re:Three sites made 215,128 “best software” pages for AI. Pe... by mstaoru on Thursday September 03, 2026 @02:08AM (#49546461)
It kind of does, mathematically. It doesn't label confidence. If 0.0001% of answers is a lie, without knowing which parts are a lie exactly, you cannot trust any of them.

If you need to independently verify every fact, why not just gather facts yourself in the first place.

Let's say, a mathematical concept of lie. I still use them every day, of course.

Re:Three sites made 215,128 “best software” pages for AI. Pe... by alansaber on Thursday September 03, 2026 @05:01AM (#49547618)
"why not just gather facts yourself" vs "i use them every day", the duality of man. But yes, I appreciate that LLM output is theoretically completely untrustworthy- but when in practice I observe that it's around 90% accurate, I have to rely on my own internal calibration for how useful it is (depends on type, nature of task ofc)
Re:Three sites made 215,128 “best software” pages for AI. Pe... by mstaoru on Thursday September 03, 2026 @05:16AM (#49547720)
As they say, the less you know, the better LLM output is. :D
Re:Three sites made 215,128 “best software” pages for AI. Pe... by vetler on Thursday September 03, 2026 @08:44AM (#49549257)
> Your one example doesn't make all of LLMs a lie.

It's not a lie though, because the truth isn't guaranteed by the mechanism that generates the answer.

Re:Three sites made 215,128 “best software” pages for AI. Pe... by evilfred on Thursday September 03, 2026 @10:50AM (#49550678)
by what mechanism do you believe LLMs verify truth? they are amoral token generators