Re:30% Drop In o1-Preview Accuracy When Putnam Problems Are ...
by empath75 on Thursday January 02, 2025 @09:57AM
(#42574947)
The comments in this thread are completely disconnected from the contents of the paper, and the thread title is rage bait and doesn't reflect the contents of the paper, either. Being able to solve a significant fraction of those problems is a pretty amazing achievement, even if it's sometimes tricked by minor variations. People are throwing around words like "fraud" or "hoax", and it's just wishcasting or whistling past the graveyard.
I think a fair conclusion here is reasoning models are still really good at solving very difficult math and competitive programming problems, but just better at ones they have seen before.