News for hackers. Stuff that matters.
Re:MAI Code 2 and Code 2 Flash by spwa4 on Tuesday November 17, 2026 @06:17PM (#50539594)
After evaluating this a bit more thoroughly ... the BIG model (not the flash one) is very comparable to qwen3-flash-next (ie. previous generation qwen flash model). Both models are significantly worse than both Qwen3.8-Max and Qwen4-Max, and are blown out of the water by Deepseek v4.1 (including the Flash version tbh):

    Benchmark            MAI-2  MAI-Flash  Qwen4-Max  Q3.8-Max  Q3.8-Next  DS-V4.1
    -------------------  -----  ---------  ---------  --------  ---------  -------
    DeepSWE 1.1          73.2     59.9       75.7      69.3*      58.7*     77.0*
    Terminal-Bench 2.1   84.6     78.3         -       86.6*      86.1*     90.6*
    SWE-Atlas            43.1     44.8         -          -          -         -
    FrontierCode 1.1       -        -          -          -          -         -
    Terminal-Bench 4.0   26.3      2.5       48.5      38.9*      25.3*     42.3*
Re:MAI Code 2 and Code 2 Flash by spwa4 on Wednesday November 18, 2026 @03:17AM (#50543337)
Wow, this model might be distilled from Qwen3-flash-next, after trying it. Certainly to an extent that's true. That would make the training cost much less if true.

I wonder if I could do something similar.