Re:In 27,280 coding agent runs, 64% of inference spend went ...
by onlyrealcuzzo on Tuesday November 17, 2026 @03:08PM
(#50537409)
There's thousands of benchmarks, and nobody can agree on what the best models are if you run all of the benchmarks which costs tens of thousands of dollars for frontier models.
Then, somebody runs a small subset of benchmarks - with not enough data to have any statistical significance - and makes further claims.
Then, somebody runs a small subset of benchmarks - with not enough data to have any statistical significance - and makes further claims.
How are we supposed to derive value from this?