Re:Legal models hallucinate in 1 out of 6 (or more) benchmar...
by empath75 on Saturday June 01, 2024 @12:58PM
(#40547327)
I mostly see people ignoring successes and playing up every error.
Re:Legal models hallucinate in 1 out of 6 (or more) benchmar...
by throw46365 on Saturday June 01, 2024 @02:28PM
(#40547983)
But the errors are fundamental, and the successes actually subjective as a result.
That is, it appears to get things right, really a lot, but the conclusions people draw about why it gets things right are undermined by the nature of the errors.
Like, it must have a world model, it must understand the meaning of... etc.; the nature of the errors they are downplaying fundamentally undermines the certainty of these projections.
I don't think I've seen a single article about an AI getting things wrong, recently, where there was a nuanced notion about whether it was actually wrong.
I don't think we're anywhere close to "nuanced mistakes are the main problem" yet.