News for hackers. Stuff that matters.
Re:Claude 2 Internal API Client and CLI by zmmmmm on Friday July 14, 2023 @06:00PM (#36730549)
> If the summaries of the test documents are good, future summaries will probably be OK too

But that is exactly what is problematic with hallucinations. It's a rare / exceptional behaviour that triggers extreme departure from reality. So you can't estimate the extremes by extrapolating from common / moderate observations. You would have to test a lot of previous documents to be confident, and even then there would be a residual risk.

Re:Claude 2 Internal API Client and CLI by sitkack on Friday July 14, 2023 @07:18PM (#36731245)
As with anyone, if you are going to take action on something, double check it.