News for hackers. Stuff that matters.
Re:Qwen4-Max by nsagent on Monday November 16, 2026 @05:52AM (#50516192)
You use something like item response theory [1] to be able to change the test over time while maintaining validity. It's what standardized tests like the GRE use.

Interestingly, Artificial Analysis does have a score that uses IRT, it's their Progress Index [2].

[1]: https://en.wikipedia.org/wiki/Item_response_theory

[2]: https://artificialanalysis.ai/methodology/progress-index

There are no comments on this comment yet.