News for hackers. Stuff that matters.
Re:Qwen3.7-Max: The Agent Frontier by Eridrus on Wednesday May 20, 2026 @05:03PM (#48214092)
Nobody releases numbers that show them to be worse than competitors lol.

This even applies to OpenAI & Anthropic who don't even eval on the same datasets a lot of the time.

Re:Qwen3.7-Max: The Agent Frontier by Ey7NFZ3P0nzAe on Thursday May 21, 2026 @03:00AM (#48218895)
I do recall mistral doing this. It's not always about being the best, but also fastest or smallest. The user should have all the information for its own use case.
Re:Qwen3.7-Max: The Agent Frontier by Eridrus on Thursday May 21, 2026 @10:34AM (#48223352)
If your model doesn't actually show the tradeoff you're getting for speed, you're doing marketing and not benchmarking.

Which is fine, we all have to make money, but it is disingenuous. It's just unfortunate that running some of these benchmarks is so expensive that it's not really realistic for most companies to actually run them.