News for hackers. Stuff that matters.
Re:OpenAI O3-Mini by godelski on Friday January 31, 2025 @05:48PM (#42893262)

  > If so, where do they indicate they failed to randomize/blind the raters?

  Win rate if user is under time constraint
This is hard to read tbh. Is it STEM? Non-STEM? If it is STEM then this shows there is a bias. If it is Non-STEM then this shows a bias. If it is a mix, well we can't know anything without understanding the split.

Note that Non-STEM is still within error. STEM is less than 2 sigma variance, so our confidence still shouldn't be that high.

There are no comments on this comment yet.