News for hackers. Stuff that matters.
Re:MAI Code 2 and Code 2 Flash by spwa4 on Tuesday November 17, 2026 @05:28PM (#50539168)
They did the trick the Chinese models didn't use. They used older versions of the Benchmarks.

According to almost the same benchmarks Qwen4 Max is Opus 5.2 level, so it would only be slightly worse than this model.

Re:MAI Code 2 and Code 2 Flash by spwa4 on Tuesday November 17, 2026 @06:17PM (#50539594)
After evaluating this a bit more thoroughly ... the BIG model (not the flash one) is very comparable to qwen3-flash-next (ie. previous generation qwen flash model). Both models are significantly worse than both Qwen3.8-Max and Qwen4-Max, and are blown out of the water by Deepseek v4.1 (including the Flash version tbh):

    Benchmark            MAI-2  MAI-Flash  Qwen4-Max  Q3.8-Max  Q3.8-Next  DS-V4.1
    -------------------  -----  ---------  ---------  --------  ---------  -------
    DeepSWE 1.1          73.2     59.9       75.7      69.3*      58.7*     77.0*
    Terminal-Bench 2.1   84.6     78.3         -       86.6*      86.1*     90.6*
    SWE-Atlas            43.1     44.8         -          -          -         -
    FrontierCode 1.1       -        -          -          -          -         -
    Terminal-Bench 4.0   26.3      2.5       48.5      38.9*      25.3*     42.3*
Re:MAI Code 2 and Code 2 Flash by spwa4 on Wednesday November 18, 2026 @03:17AM (#50543337)
Wow, this model might be distilled from Qwen3-flash-next, after trying it. Certainly to an extent that's true. That would make the training cost much less if true.

I wonder if I could do something similar.