News for hackers. Stuff that matters.
Re:30% Drop In o1-Preview Accuracy When Putnam Problems Are ... by bcoates on Wednesday January 01, 2025 @04:56PM (#42569642)
This is why RLHF causes those overly verbose answers to simple questions, it's a fundamentally busted evaluation function so you wind up optimizing for the wrong thing
Re:30% Drop In o1-Preview Accuracy When Putnam Problems Are ... by gus_massa on Friday January 03, 2025 @09:13PM (#42591677)
[Sorry for the dealy.]

In one extreme there are wall of text and in the other extreme very short answers that only the initiated understand (like inside jokes). Somewhere in between there is a sweet spot that helps everyone else to follow the discusion and gain a litle of knowdledge.

(I don't claim I get the best lenght in my comments, but I hope it's good enough.)