Nice! I'm not surprised at your findings (anymore). Mechanical reliability is the key to small models, and it's a big unlock. I've seen the same thing you just described. And the agnostic nudges forge sends at inspired by exactly that. Just show the model how it failed, gracefully, and it'll likely figure a way out of it itself.
Forge doesn't have a SWE-specific eval, but I've built a custom coding harness (not public yet but maybe soon) built on forge and saw the same behavior you seem to have seen in agentic coding.
Re:
by jeffreygoesto on Wednesday May 20, 2026 @01:49AM
(#48203583)
Reads like processes guarding mediocre teams into higher probability of success? I can hear Alanis Morissette in my head now somehow...
Basically, yeah...this uplifts everything I've tested, but it's the small models that benefit most. A perfect model would get no benefit.
Mostly, I'm embarrassed I've done this whole public reveal without any use of Alanis Morissette anywhere in the work :/
One of the most surprising findings was when a 9B model self-corrected through 4 tool parse failures within the guard rails. It tried to use a complex tool (patch_file), kept failing and eventually downshifted to a simpler tool (edit_line) that it could actually execute. The guardrails didn't make the model smarter, it just narrowed the execution space until it could find something that worked.
Brief: https://statewright.ai/research