News for hackers. Stuff that matters.
Re:Claude Sonnet 4.5 by zamadatix on Monday September 29, 2025 @08:37PM (#45420647)
> The benchmarks are not typically ongoing, we do not often see comparisons between week 1 and week 8

You don't need to compare "A (Week 1)" to "A (Week 8)" to be able to show "B (Week 1)" is genuinely x% better than "A (Week 1)".

Re:Claude Sonnet 4.5 by MichealCodes on Monday September 29, 2025 @09:23PM (#45420949)
As I said sprinkle a bit of benchmarks polluting the training and you have your loop. Each iteration will be better at benchmarks if that's the goal and that goal/context reinforces.
Re:Claude Sonnet 4.5 by zamadatix on Tuesday September 30, 2025 @01:04PM (#45428125)
Sprinkling in benchmark training isn't a loop, it's just plain cheating. Regardless, not all of these benchmarks are public and, even with mass collusion across the board, it wouldn't make sense only open weight LLMS have been improving.