Re:30% Drop In o1-Preview Accuracy When Putnam Problems Are ...
by bcoates on Wednesday January 01, 2025 @04:56PM
(#42569642)
This is why RLHF causes those overly verbose answers to simple questions, it's a fundamentally busted evaluation function so you wind up optimizing for the wrong thing
[ Parent
| on HN ]
Re:30% Drop In o1-Preview Accuracy When Putnam Problems Are ...
by gus_massa on Friday January 03, 2025 @09:13PM
(#42591677)
[Sorry for the dealy.]
In one extreme there are wall of text and in the other extreme very short answers that only the initiated understand (like inside jokes). Somewhere in between there is a sweet spot that helps everyone else to follow the discusion and gain a litle of knowdledge.
(I don't claim I get the best lenght in my comments, but I hope it's good enough.)