Code has a built-in form of easy fact checking, which makes it one of the most appropriate applications for LLMs. It's much harder to spot a hallucinated fact in a paragraph of prose than it is to spot a hallucinated API method.
The skills you most need to develop in order to take advantage of LLM assisted programming are code reading, code review, manual and automated testing and being really good at thinking of edge-cases that might not be covered.
It turns out these are important skills for being a great developer already - LLMs just force the issue on them a little more.
So I keep wondering if we just save time by introducing more unknown bugs using GPT?
I guess this also has a lot to do with what code is written. I would be much more concerned with a system level C++ library than some JavaScript CRUD.
Gpt-4 often handles errors well. The generated code is easy to review if you understand what you asked for (if it generates tests and examples too-which it can). Etc
I'm sure there are good use cases, but I just haven't seen anything life-changing in my experiments with various LLM products. The only thing I use day to day is IntelliCode, which saves a bit of typing here and there.
Or if it hallucinates a method name, it might be in a code path that goes untested. How often are people using these tools to also write comprehensive test suites?
If you don't do that, the chances LLM-generated code will introduce weird bugs are high. But the chances you yourself (or your coworkers) will introduce weird bugs are high as well - QA, code reviews and testing are important no matter what helped write the code.
What about when it confabulates a module name that's been squatted by a malware distributor?
Just like if someone opens a PR against your project on GitHub you should review the dependencies they are trying to bring in as well.
I often see GPT/Gemini propose code solutions that refer to non-existing libraries or methods inside of these libraries. Perhaps the solution is to use specialized AI for coding is more advised.
And dont use the generic ones. However these hallucinations are still facts. The issue is that humans are also prone to hallucinations; this is perhaps the most challenging aspect to solve in AI. Everything the AI says must be vetted/fact-checked.
Back to coding with AI: i noticed AI, as described by the original author, is perhaps best integrated as an assistant. It works well when given enough context and a simple task that could otherwise take hours to complete but is now in seconds. So think of:
- given this function: …
- write test cases that covers the most common inputs.
LLM’s are still long way to go to replace programmers. But at the pace it’s going, it feels scary sometimes.
If a robot could paint your house, but made three small errors, would you refuse to use it? Or would you just fix the three small errors by painting over them?
There's some kind of John Henry complex going on in this AI discussion.
I've worked a good number of hours with Claude Opus and it has never produced non-compiling code (ChatGPT 4 does that for me), but it can create quite subtle bugs, which is missed by the "just make sure it compiles"-type comments in this thread.
So...why should they be included?
I really worry about this "often wrong" part - you only know they are wrong if you already know what you're doing. Otherwise you end up trying to use hallucinated APIs & libraries, or produce code not better than copy & pasting StackOverflow answers (which is what the AI was trained on anyway).