Re:62% of code generated by GPT-4 contains API misuses
by CharlesW on Sunday August 27, 2023 @08:24PM
(#37288145)
> There simply aren't any tools that can plug into a massive codebase with millions of lines.
If Sourcegraph's offering (https://about.sourcegraph.com/cody) has limits I'm not aware of, this is just a matter of time. But in my experience, this is a "nice to have" rather than a "must have" when it comes to the benefits of GenAI as it applies to coding.
Re:62% of code generated by GPT-4 contains API misuses
by ianbutler on Sunday August 27, 2023 @08:22PM
(#37288128)
Yeah I think this is really one of the key issues. I found all of this tech to be really great with my side projects and really lacking with my big corpo codebase fragmented across multiple teams, projects and libraries.
Re:62% of code generated by GPT-4 contains API misuses
by version_five on Sunday August 27, 2023 @08:54PM
(#37288365)
I'd go further and say the sweet spot is probably writing a simple function that fits in the model's context window, in a language you don't know. Even here it screws up, but you can go back and forth and get somewhere. I can only imagine the mess one would make using it to build an actual application of any size. The analogy that comes to mind is using google translate to try and write an essay.
Re:62% of code generated by GPT-4 contains API misuses
by plorkyeran on Sunday August 27, 2023 @08:59PM
(#37288407)
I do think that's the big disconnect. There's jobs where you basically only ever create brand new code from scratch - writing lots of little scripts, creating new screens in a webapp, or whatever - and current AI tools can save a ton of time for that. There's also jobs where it's normal to spend a week figuring out what to change and then five minutes changing it, and current AI tools are worthless for that. Most jobs are somewhere in between those extremes, but the hype is coming from people pretty far on the "mostly writing new code" end.
I am in the latter group, and I don't find it all that helpful. There simply aren't any tools that can plug into a massive codebase with millions of lines. I never just work on one specific repository either - a change generally involves multiple repositories.
On the other hand, if you're writing a smaller standalone utility, or working on a greenfield application, it can be helpful (with all the useful caveats about hallucations).
I think that a lot of ML folks, especially those in research, fall into the former category, which is why there has been so much hype.