News for hackers. Stuff that matters.
Re:OpenAI O3-Mini by janalsncm on Friday January 31, 2025 @04:16PM (#42892123)
> I'm usually just selecting the one that answered first

Which is why you randomize the order. You aren’t a tester.

56% vs 44% may not be noise. That’s why we have p values. It depends on sample size.

Re:OpenAI O3-Mini by jhardy54 on Saturday February 01, 2025 @01:37AM (#42896257)
The order doesn't matter. They often generate tokens at different speeds, and produce different lengths of text. "The one that answered first" != "The first option"