After evaluating this a bit more thoroughly ... the BIG model (not the flash one) is very comparable to qwen3-flash-next (ie. previous generation qwen flash model). Both models are significantly worse than both Qwen3.8-Max and Qwen4-Max, and are blown out of the water by Deepseek v4.1 (including the Flash version tbh):
Benchmark MAI-2 MAI-Flash Qwen4-Max Q3.8-Max Q3.8-Next DS-V4.1
------------------- ----- --------- --------- -------- --------- -------
DeepSWE 1.1 73.2 59.9 75.7 69.3* 58.7* 77.0*
Terminal-Bench 2.1 84.6 78.3 - 86.6* 86.1* 90.6*
SWE-Atlas 43.1 44.8 - - - -
FrontierCode 1.1 - - - - - -
Terminal-Bench 4.0 26.3 2.5 48.5 38.9* 25.3* 42.3*Wow, this model might be distilled from Qwen3-flash-next, after trying it. Certainly to an extent that's true. That would make the training cost much less if true.
I wonder if I could do something similar.
According to almost the same benchmarks Qwen4 Max is Opus 5.2 level, so it would only be slightly worse than this model.