Re:Qwen3.6-Max-Preview: Smarter, Sharper, Still Evolving
by fwipsy on Monday April 20, 2026 @07:39PM
(#47842625)
Perhaps not even necessarily subjective, just performance is highly task-dependent and even variable within tasks. People get objectively different experiences, and assume one or another is better, but it's basically random.
[ Parent
| on HN ]
Re:Qwen3.6-Max-Preview: Smarter, Sharper, Still Evolving
by operatingthetan on Monday April 20, 2026 @07:58PM
(#47842819)
>just performance is highly task-dependent and even variable within tasks. People get objectively different experiences, and assume one or another is better, but it's basically random.
You are right that this is not exactly subjectivity, but I think for most people it feels like it. We don't have good benchmarks (imo), we read a lot about other people's experiences, and we have our own. I think certain models are going to be objectively better at certain tasks, it's just our ability to know which currently is impaired.
Re:Qwen3.6-Max-Preview: Smarter, Sharper, Still Evolving
by easygenes on Tuesday April 21, 2026 @12:06AM
(#47844404)
Unless you're looking at something like a pass@100 benchmark, the benchmarks are confounded heavily by a likelihood of a "golden path" retrieval within their capabilities. This is on top of uncertainties like how well your task within a domain maps to the relevant test sets, as well as factors like context fullness and context complexity (heavy list of relevant complex instructions can weigh on capabilities in different ways than e.g. having a history where there's prior unrelated tasks still in context).
The best tests are your own custom personal-task-relevant standardized tests (which the best models can't saturate, so aiming for less than 70% pass rate in the best case).
All this is to say that most people are not doing the latter and their vibes are heavily confounded to the point of being mostly meaningless.
Re:Qwen3.6-Max-Preview: Smarter, Sharper, Still Evolving
by mentalgear on Tuesday April 21, 2026 @05:06PM
(#47854506)
This. Plus if you want to even attempt measuring real 'intelligence' you want to run a neuro-symbolic, de-lexicalized benchmark (e.g. DL-ReasonSuite, SoLT, GSM-Symbolic) - which none of the providers releasing new models showcase.
Re:Qwen3.6-Max-Preview: Smarter, Sharper, Still Evolving
by make3 on Wednesday April 22, 2026 @12:41AM
(#47859100)
The pass@100 is such a weird critique angle that is surprisingly mainstream; guess what, no one cares if the correct answer is in the top 100, it needs to be the top 1. A model with a better answer in the top 1 is a better model, full stop.