News for hackers. Stuff that matters.

A comment on The Deep Research Problem. Parent comment

Re:The Deep Research Problem by SubiculumCode on Tuesday February 25, 2025 @11:30PM (#43180612)
In some ways, it's a good tool to teach yourself to sus out the real clues to reliability, not format and authoritative tone.
[ Parent | on HN ]
Re:The Deep Research Problem by j_maffe on Wednesday February 26, 2025 @03:56AM (#43182025)
But that's the thing. The only way to truly find out if it's reliable (>90%) is to check the data yourself.
[ Parent | on HN ]
Re:The Deep Research Problem by SubiculumCode on Wednesday February 26, 2025 @11:09AM (#43184995)
This is why metrics and leaderboards like these are so important (but under reported on): https://github.com/vectara/hallucination-leaderboard https://www.kaggle.com/facts-leaderboard

Google Gemni models seem to lead...hopefully the metrics aren't being gamed.

[ Parent | on HN ]