Re:Legal models hallucinate in 1 out of 6 (or more) benchmar...
by empath75 on Friday May 31, 2024 @03:43PM
(#40539411)
I mostly see the reverse.
Re:Legal models hallucinate in 1 out of 6 (or more) benchmar...
by throw46365 on Friday May 31, 2024 @04:11PM
(#40539695)
You mostly see people projecting perceived error onto LLMs?
I don't think I've seen a single article about an AI getting things wrong, recently, where there was a nuanced notion about whether it was actually wrong.
I don't think we're anywhere close to "nuanced mistakes are the main problem" yet.
Re:Legal models hallucinate in 1 out of 6 (or more) benchmar...
by empath75 on Saturday June 01, 2024 @12:58PM
(#40547327)
I mostly see people ignoring successes and playing up every error.
Re:Legal models hallucinate in 1 out of 6 (or more) benchmar...
by throw46365 on Saturday June 01, 2024 @02:28PM
(#40547983)
But the errors are fundamental, and the successes actually subjective as a result.
That is, it appears to get things right, really a lot, but the conclusions people draw about why it gets things right are undermined by the nature of the errors.
Like, it must have a world model, it must understand the meaning of... etc.; the nature of the errors they are downplaying fundamentally undermines the certainty of these projections.
Thank you for this sentence: it is hard to get across how often Gen-AI proponents are actually projecting perceived success onto LLMs while downplaying error.