News for hackers. Stuff that matters.
Re:Mistral Releases Devstral 2 (72.2% SWE-Bench Verified) an... by vikramkr on Tuesday December 09, 2025 @11:44PM (#46214172)
If the thing we're measuring is a the ability to write code, visually reason, and handle extrapolating to out of sample prompts, then why shouldn't we evaluate it by asking it to write code to generate a strange image that it wouldn't have seen in its training data?

There are no comments on this comment yet.