News for hackers. Stuff that matters.

A comment on AutoCodeRover resolves 22% of real-world GitHub issues in SWE-bench lite. Parent comment

Re:AutoCodeRover resolves 22% of real-world GitHub issues in... by ectopasm83 on Tuesday April 09, 2024 @08:10AM (#39978565)
The point is that the success rate is progressing, paper after paper

> The baseline results of Magis (10%), Devin (14%) are evaluated in another subset of SWE-bench, which we cannot directly compare with, so we take the results from their technical reports as a reference.

Wondering how it compares with these models.

[ Parent | on HN ]
Re:AutoCodeRover resolves 22% of real-world GitHub issues in... by invalidusernam3 on Tuesday April 09, 2024 @09:50AM (#39979428)
Why not use AutoCodeRover, Magis, and Devin together for 46%

/s

[ Parent | on HN ]