You need an adversarial co-pilot to write tests for those tests so you would put the two AIs against each other to try to properly test.
I've used Copilot to help with writing verbose unit tests. It can do it as long as you keep an eye over it (basically like an autocomplete), it definitely cannot produce robust test cases on its own though. If you try to do that, it won't take "meta-tests" to figure out they don't look right.
Re:GitHub Copilot is generally available
by DeathArrow on Tuesday June 21, 2022 @03:42PM
(#31828239)
>Can Copilot write tests? That way it could test its own code and tweak it until it works.
Sure it can. But you can't rely on them being good. You have to read the tests carefully.
Re:GitHub Copilot is generally available
by datatrashfire on Tuesday June 21, 2022 @04:13PM
(#31828602)
I can almost envision a future where human devs write tests, code generating frameworks build code from a spec.
Re:GitHub Copilot is generally available
by paskozdilar on Tuesday June 21, 2022 @04:17PM
(#31828633)
I think that's the underlying idea of Logic Programming.
And UML diagrams.
Re:GitHub Copilot is generally available
by Hnrobert42 on Tuesday June 21, 2022 @05:24PM
(#31829283)
Isn’t this how they managed to stop the Borg?
Of course, one would then ask how to verify tests. I suppose Copilot could write meta-tests - tests that verify other tests. That way it could test its own tests and tweak them until they work.
Of course, one would then ask how to verify meta-tests. I suppose Copilot could write meta-meta-tests - tests that verify meta-tests. That way it could test its own meta-tests and tweak them until they work.
Of course, one would then ask how to verify meta-meta-tests...