News for hackers. Stuff that matters.
Re:Qwen3.7-Max: The Agent Frontier by goyozi on Wednesday May 20, 2026 @07:10AM (#48205897)
These are very good numbers. I still don’t get why they don’t compare against latest competitor versions in these posts, it’s not like we’re all not going to notice.
Re:Qwen3.7-Max: The Agent Frontier by hmokiguess on Wednesday May 20, 2026 @08:15AM (#48206459)
this puzzles me too, I want to know
Re:Qwen3.7-Max: The Agent Frontier by htrp on Wednesday May 20, 2026 @08:32AM (#48206653)
I think its part of the expectation setting (with a side of we did our distillation/ eval harness on a specific model).

if they say it's 4.7 comparable, it anchors that into your head as the model to evaluate against.

Re:Qwen3.7-Max: The Agent Frontier by maelito on Wednesday May 20, 2026 @08:36AM (#48206695)
Marketing.
Re:Qwen3.7-Max: The Agent Frontier by Aurornis on Wednesday May 20, 2026 @08:57AM (#48206940)
I think the argument is that trying to suggest that they’re close to N months from SOTA.

Realistically I assume they hope readers don’t notice the fine details.

The Qwen models are great for open weights but for every past release they haven’t performed as well as the benchmarks in my experience. They’re optimizing for benchmark numbers because they know it works.

Re:Qwen3.7-Max: The Agent Frontier by epolanski on Wednesday May 20, 2026 @09:39AM (#48207585)
> Realistically I assume they hope readers don’t notice the fine details.

The pool of people reading such articles while ignoring such details can't be big.

Re:Qwen3.7-Max: The Agent Frontier by Aurornis on Wednesday May 20, 2026 @09:42AM (#48207645)
I disagree. Most people skim articles, not read them deeply.

On Hacker News I wonder if most people even opened the article at all most times.

Re:Qwen3.7-Max: The Agent Frontier by hadlock on Wednesday May 20, 2026 @01:49PM (#48211417)
Slashdot coined RTFA in the 90s, what you're suggesting isn't a new concept by any measure

e: which itself is a modification of RTFM from usenet

Re:Qwen3.7-Max: The Agent Frontier by beydogan on Wednesday May 20, 2026 @09:12AM (#48207129)
honestly, initial version of Opus-4.6 was much better than whatever we are being served right now as 4.7. If it performs same level to that, i'm totally willing to switch.
Re:Qwen3.7-Max: The Agent Frontier by hypercube33 on Wednesday May 20, 2026 @10:09AM (#48208107)
4.6 was an awful experience the month I used it right after launch where it didn't ask anything just made assumptions and went on its merry way. 4.5 and 4.7 don't do that for me but 4.7 eats my quota for breakfast so I've been avoiding using it because I like to have it for more than an hour a day.
Re:Qwen3.7-Max: The Agent Frontier by goyozi on Wednesday May 20, 2026 @10:44AM (#48208741)
I feel like I had the best and worst ~month experience on 4.6. Initially when it came out, it seemed to ask good questions and genuinely do well on complex tasks. From about mid-March it was absolutely abysmal, it seemed to assume the stupidest answer/angle for everything and make weird mistakes. 4.7 seems decent so far but usage hurts - at some point my company switched me to standard seat and I used up 80% of my session usage in 1 prompt. I got my premium seat back since but I think pro/standard plan + opus 4.7 is unusable for daily driving.
Re:Qwen3.7-Max: The Agent Frontier by verdverm on Wednesday May 20, 2026 @11:24AM (#48209299)
That experience is also likely tied to the claude harness around the model, and not being as tuned right after model release. They iterate on this and different models need different words (unfortunately...).
Re:Qwen3.7-Max: The Agent Frontier by NiloCK on Wednesday May 20, 2026 @09:14AM (#48207155)
I find it forgivable if it's within minor version bump. (NB that x.5 is now a defacto major-version bump for LLMs for whatever reason).

Even with LLMs, posts like this don't just fall out of a coconut tree. If you have a set of target benchmarks for your own model, then keeping "the set" of side-by-side comparable models is its own maintenance headache.

Re:Qwen3.7-Max: The Agent Frontier by Eridrus on Wednesday May 20, 2026 @05:03PM (#48214092)
Nobody releases numbers that show them to be worse than competitors lol.

This even applies to OpenAI & Anthropic who don't even eval on the same datasets a lot of the time.

Re:Qwen3.7-Max: The Agent Frontier by Ey7NFZ3P0nzAe on Thursday May 21, 2026 @03:00AM (#48218895)
I do recall mistral doing this. It's not always about being the best, but also fastest or smallest. The user should have all the information for its own use case.
Re:Qwen3.7-Max: The Agent Frontier by Eridrus on Thursday May 21, 2026 @10:34AM (#48223352)
If your model doesn't actually show the tradeoff you're getting for speed, you're doing marketing and not benchmarking.

Which is fine, we all have to make money, but it is disingenuous. It's just unfortunate that running some of these benchmarks is so expensive that it's not really realistic for most companies to actually run them.