30% Drop In o1-Preview Accuracy When Putnam Problems Are Slightly Variated

Re:30% Drop In o1-Preview Accuracy When Putnam Problems Are ... by jsheard on Wednesday January 01, 2025 @10:42AM (#42566672)

To be clear I was testing with 4o, good to know that o1 has a better grasp of basic arithmetic. Regardless my point was less to do with the models ability to do math and more to do with OpenAI seeming to cover up its lack of ability.

[ Parent | on HN ]

Re:30% Drop In o1-Preview Accuracy When Putnam Problems Are ... by whimsicalism on Wednesday January 01, 2025 @10:56AM (#42566767)

i think it’s mostly that o1 mini can think through the solution before it starts writing the poem.

i’m able to reproduce your failure on 4o

[ Parent | on HN ]