Re:AutoCodeRover resolves 22% of real-world GitHub issues in...
by yuntong on Tuesday April 09, 2024 @10:48AM
(#39980047)
The previous comment was to describe the experiment settings.
AutoCodeRover currently generates patches. Auto-generating high quality tests can be a parallel effort and another direction to explore. These efforts can eventually be used together.
Re:AutoCodeRover resolves 22% of real-world GitHub issues in...
by dboreham on Tuesday April 09, 2024 @03:55PM
(#39983542)
The point is that a patch without a test is not generally a useful thing. How do we know the AI generated patches are valid?
Re:AutoCodeRover resolves 22% of real-world GitHub issues in...
by nfm on Tuesday April 09, 2024 @04:20PM
(#39983801)
I agree in principle, but if it also generated a test, how would you know that was valid?
The value I get from copilot is the ability to code faster, not the ability to code.
Re:AutoCodeRover resolves 22% of real-world GitHub issues in...
by abhikrc on Wednesday April 10, 2024 @01:38AM
(#39987306)
We know that by testing the patches. Automated Program Repair is an area of research which has done this.
The patches are tested against a test-suite. So, if there are tests, we welcome them, we definitely use them to validate the patches.
You have this backwards : it's traditional (at least in the past 15 years or so) to have a test to go along with every code change. The idea is that the test proves a) the bug existed prior to the fix and b) the bug is not there after the fix is applied. Commenters here are noting that ACR generates fixes but not tests.