News for hackers. Stuff that matters.
Re:Kimi K3.1 Pro and K3.1 Pro Highspeed by augment_me on Friday October 23, 2026 @03:44PM (#50214353)
Do you have an alternative? Its a benchmark collection. When a benchmark gets saturated, they remove it from the index and add new ones.

I think lm-arena is a better concept that reflects the actual target users, but it only captures chat, and relies on users spending their time improving it.

Re:Kimi K3.1 Pro and K3.1 Pro Highspeed by genneth on Friday October 23, 2026 @04:06PM (#50214580)
https://epoch.ai/eci?view=graph&tab=release-date is at least an alternative. They are slower to update, but entirely anecdotally I find the numbers are aligned with my personal observations a bit better than AA. But I am also highly aware this is some pretty confirmation-bias heavy ares of meta-benchmark picking...
Re:Kimi K3.1 Pro and K3.1 Pro Highspeed by augment_me on Friday October 23, 2026 @04:19PM (#50214751)
If we ignore the older releases (sonnet 5, etc that exist in that one) and the new models epoch has not benchmarked, we end up with the same order within the noise brackets as in ArtificialAnalysis.

I do trust epoch more than AA due to capital incentives, but in this case I don't think like there is a difference