Re:Qwen3.6-Max-Preview: Smarter, Sharper, Still Evolving
by mentalgear on Tuesday April 21, 2026 @05:06PM
(#47854506)
This. Plus if you want to even attempt measuring real 'intelligence' you want to run a neuro-symbolic, de-lexicalized benchmark (e.g. DL-ReasonSuite, SoLT, GSM-Symbolic) - which none of the providers releasing new models showcase.
Re:Qwen3.6-Max-Preview: Smarter, Sharper, Still Evolving
by make3 on Wednesday April 22, 2026 @12:41AM
(#47859100)
The pass@100 is such a weird critique angle that is surprisingly mainstream; guess what, no one cares if the correct answer is in the top 100, it needs to be the top 1. A model with a better answer in the top 1 is a better model, full stop.
The best tests are your own custom personal-task-relevant standardized tests (which the best models can't saturate, so aiming for less than 70% pass rate in the best case).
All this is to say that most people are not doing the latter and their vibes are heavily confounded to the point of being mostly meaningless.