News for hackers. Stuff that matters.

A comment on Claude Opus 4.5. Parent comment

Re:Claude Opus 4.5 by dash2 on Monday November 24, 2025 @11:40PM (#46042385)
It's not non-empirical. He was careful to give it the same experiment twice. The dependent variable is his judgment, sure, but why shouldn't we trust that if he's an experienced SWE?
[ Parent | on HN ]
Re:Claude Opus 4.5 by atq2119 on Tuesday November 25, 2025 @08:24AM (#46045558)
Sample size is way too small.

Unless he was able to sample with temperature 0 (and get fully deterministic results both times), this can just be random chance. And experience as SWE doesn't imply experience with statistics and experiment design.

[ Parent | on HN ]