News for hackers. Stuff that matters.
Re:30% Drop In o1-Preview Accuracy When Putnam Problems Are ... by jorvi on Wednesday January 01, 2025 @05:56PM (#42569997)
See my more detailed upthread response here: https://news.ycombinator.com/item?id=42569937

But, like Zahlman points out, its a trick question, and instead of admitting it doesn't know or even prepending "I don't know for sure, but:", it just burps up its best-effort answer. There is no one spirit that represents The Netherlands. If a LLM is so good it "oneshots any question", it should realize it doesn't have a unanimous answer and tell me.

There are no comments on this comment yet.