Re:30% Drop In o1-Preview Accuracy When Putnam Problems Are ...
by _heimdall on Thursday January 02, 2025 @10:29PM
(#42582050)
> I think we're in agreement? I'm saying their measure in this case is no worse than any other, but not that it's a fundamental truth.
Maybe we are? I think I lost the thread a bit here.
> chess, Jeopardy, composing music, painting, maths, languages, passing medical or law degrees
That's interesting, I would have still chalked skill in those areas as a sign of intelligence and didn't realize most people wouldn't once AI (or ML) could do it. To me an AI/LLM/ML being good at those is at least a sign that they have gotten good at mimicking intelligence if nothing else, and a sign that we really are getting out over our skis risking these tools without knowing how they really work.
Maybe we are? I think I lost the thread a bit here.
> chess, Jeopardy, composing music, painting, maths, languages, passing medical or law degrees
That's interesting, I would have still chalked skill in those areas as a sign of intelligence and didn't realize most people wouldn't once AI (or ML) could do it. To me an AI/LLM/ML being good at those is at least a sign that they have gotten good at mimicking intelligence if nothing else, and a sign that we really are getting out over our skis risking these tools without knowing how they really work.