Claude 2 Internal API Client and CLI

Re:Claude 2 Internal API Client and CLI by ronsor on Friday July 14, 2023 @04:53PM (#36729789)

Presumably one can test it with documents one has already read and knows before. If the summaries of the test documents are good, future summaries will probably be OK too.

[ Parent | on HN ]

Re:Claude 2 Internal API Client and CLI by lambdaba on Friday July 14, 2023 @05:17PM (#36730051)

Maybe having it summarize a fiction book (outside of training data)?

[ Parent | on HN ]

Re:Claude 2 Internal API Client and CLI by zmmmmm on Friday July 14, 2023 @06:00PM (#36730549)

> If the summaries of the test documents are good, future summaries will probably be OK too

But that is exactly what is problematic with hallucinations. It's a rare / exceptional behaviour that triggers extreme departure from reality. So you can't estimate the extremes by extrapolating from common / moderate observations. You would have to test a lot of previous documents to be confident, and even then there would be a residual risk.

[ Parent | on HN ]

Re:Claude 2 Internal API Client and CLI by sitkack on Friday July 14, 2023 @07:18PM (#36731245)

As with anyone, if you are going to take action on something, double check it.

[ Parent | on HN ]