News for hackers. Stuff that matters.

A comment on 30% Drop In o1-Preview Accuracy When Putnam Problems Are Slightly Variated. Parent comment

Re:30% Drop In o1-Preview Accuracy When Putnam Problems Are ... by bwfan123 on Wednesday January 01, 2025 @01:41PM (#42568066)
this work is similar to the GSM symbolic paper (applied to putnam) https://arxiv.org/html/2410.05229v1

going forward, llm performance must be reported on the confounded benchmark as well

[ Parent | on HN ]

There are no comments on this comment yet.