News for hackers. Stuff that matters.
Re:How to generate tested software packages using LLMs, a sa... by travisjungroth on Tuesday August 08, 2023 @02:00AM (#37045061)
Based on my experience with Autopilot and Copilot, I think this is way less of a problem in code.

You can put code mistakes on a gradient, from subtle to obvious. Obvious bugs are like when the LLM finds a pattern and repeats it for 100 lines. Subtle mistakes are like misnaming a variable so you use one left over from earlier, not the correct one.

Obvious mistakes are easy to catch because they’re obvious. The LLM makes more of those. I think because of the way LLMs work, I have never seen Copilot make a subtle mistake that I wouldn’t expect of a person. People are so good at making surprising bugs, it’s really hard for Copilot to beat it.

There are no comments on this comment yet.