News for hackers. Stuff that matters.

A comment on AutoCodeRover resolves 22% of real-world GitHub issues in SWE-bench lite. Parent comment

Re:AutoCodeRover resolves 22% of real-world GitHub issues in... by DalasNoin on Tuesday April 09, 2024 @08:29AM (#39978699)
Just about a week ago open devin got about 13% on this benchmark. Just give it a few more weeks.

edit: apparently it's not the exact same benchmark but a similar one

[ Parent | on HN ]
Re:AutoCodeRover resolves 22% of real-world GitHub issues in... by moolcool on Tuesday April 09, 2024 @08:55AM (#39978901)
If it continues at this pace, then it'll solve 108% of GitHub issues in just 3 months
[ Parent | on HN ]