Re:Senior SWE-Bench: open-source benchmark that assesses age...
by LiamPowell on Friday July 03, 2026 @01:08AM
(#48771005)
I suspected as much, and that brings us to the second issue where if we use a cohort of judges then the model that likes it's own code the most still wins.
There have been studies that showed that models tended to rate responses from their own family of models better than equivalent responses from other families, eg. gpt-4 would prefer a response from gpt-3