News for hackers. Stuff that matters.

A comment on Show HN: Bullshit Detector – agent skills that fact-check videos and articles. Parent comment

Re:Show HN: Bullshit Detector – agent skills that fact-check... by simonw on Wednesday July 29, 2026 @10:17AM (#49097879)
Something I've found surprisingly effective is telling ChatGPT to "use credible sources" - you can then watch its thinking trace and see it do things like ruling out random blogs, considering media publications with a good reputation for fact checking, and double-checking information that seems unlikely.
[ Parent | on HN ]
Re:Show HN: Bullshit Detector – agent skills that fact-check... by westoque on Wednesday July 29, 2026 @10:26AM (#49097972)
I don't think random blogs should be ruled out. Just because a blog is random doesn't mean its untrue.
[ Parent | on HN ]
Re:Show HN: Bullshit Detector – agent skills that fact-check... by simonw on Wednesday July 29, 2026 @10:43AM (#49098203)
The reputation of the writer matters a lot. Recent models seem to be pretty good at evaluating author credibility as well.

What's dangerous is treating information from an anonymous author as factual. You gotta check that in other ways.

[ Parent | on HN ]
Re:Show HN: Bullshit Detector – agent skills that fact-check... by bulbar on Wednesday July 29, 2026 @10:30AM (#49098021)
It should not be seen a credible source though.

It can be, but that's hard to correctly judge on at scale.

[ Parent | on HN ]
Re:Show HN: Bullshit Detector – agent skills that fact-check... by skorniienko on Wednesday July 29, 2026 @10:19AM (#49097894)
Love it, I'll run tests for "credible sources" and will compare the results. Could be a first feature-request implemented!
[ Parent | on HN ]
Re:Show HN: Bullshit Detector – agent skills that fact-check... by skorniienko on Wednesday July 29, 2026 @12:20PM (#49099499)
Added this as issue so I don't forget https://github.com/SerhiiKorniienko/bullshit-detector/issues...
[ Parent | on HN ]
Re:Show HN: Bullshit Detector – agent skills that fact-check... by vunderba on Wednesday July 29, 2026 @04:23PM (#49102554)
For any LLM with web tool usage I’ll often suggest that it pull relevant papers from arXiv, with an expectation that it cite sources. I find that this helps improve accuracy and lets me go deeper into the material if I want to.
[ Parent | on HN ]
Re:Show HN: Bullshit Detector – agent skills that fact-check... by skorniienko on Thursday July 30, 2026 @11:57AM (#49111841)
Hi, ran the experiment with "credible sources". Results we unexpected but not less interesting. Few things worth noting: - same search query ran minutes apart return different sources, sometimes with only 1-2 overlaps. - 16 high-credibility news outlets are rejecting agent's access to a content. (like Reuters, NYT, Guardian), but all .gov, dedicated fact-checkers are working.

Detailed report, if you are interested: https://github.com/SerhiiKorniienko/bullshit-detector/blob/m...

[ Parent | on HN ]