Re:Qodo CLI agent scores 71.2% on SWE-bench Verified
by whymauri on Tuesday August 12, 2025 @10:22AM
(#44876563)
I feel like the bash only SWE Bench Verified (a.k.a model + mini-swe-agent) is the closest thing to measuring the inherent ability of the model vs. the scaffolding.
Re:Qodo CLI agent scores 71.2% on SWE-bench Verified
by NitpickLawyer on Tuesday August 12, 2025 @11:01AM
(#44877136)
There's swe-rebench, where they take "bugs/issues" by date, and you can drag a slider on their top scores to see issues solved after the model was released (obviously only truly working for open models).
I think that the next step is getting an official "checked" mark by the SWE bench team