News for hackers. Stuff that matters.
Re:Claude 3 model family by lukev on Monday March 04, 2024 @01:01PM (#39593663)
This doesn't pass the sniff test for me. Not sure if these models are memorizing the answers or something else, but it's simply not the case that they're as capable as a domain expert (yet.)

I do not have a PhD, but in areas I do have expertise, you really don't have to push these models that hard to before they start to break down and emit incomplete or wrong analysis.

Re:Claude 3 model family by matchagaucho on Monday March 04, 2024 @01:54PM (#39594378)
They claim the model was grounded with a 25-shot Chain-of-Thought (CoT) prompt.
Re:Claude 3 model family by p1esk on Monday March 04, 2024 @01:34PM (#39594104)
Have you tried the Opus model specifically?
Re:Claude 3 model family by CamperBob2 on Tuesday March 05, 2024 @02:40PM (#39608262)
The idea is that they aren't as capable as a domain expert, but they are more capable than an expert from a different domain.

E.g., if you ask a chemistry PhD something about genetics or astrophysics, you're more likely to get a correct answer from the model. Which is pretty interesting IMO.