Re:Claude Opus 5
by 2001zhaozhao on Friday July 24, 2026 @02:26PM
(#49039743)
It is probably from randomness. The benchmark tasks nowadays are so long that you can't really afford to run a large number of samples of them per model & effort combination
[ Parent
| on HN ]
Re:Claude Opus 5
by artemisart on Friday July 24, 2026 @03:39PM
(#49040678)
No it's a mean of 5 runs.
> We report FrontierCode’s overall score, a composite measure that grades each patch on blocking functional criteria (held-out unit tests) together with weighted code-quality rubric criteria, as mean@5.
They don't explain more in the system card, I guess higher effort levels could loose points on the code quality / scope / style / maintainability stuff?