62% of code generated by GPT-4 contains API misuses

Re:62% of code generated by GPT-4 contains API misuses by jojobas on Sunday August 27, 2023 @07:41PM (#37287780)

Garbage in, garbage out. I wonder if it can contextualize the code in StackOverflow questions (i.e. why doesn't this work?) as bad and that in highly rated answers as good, or "code is code".

[ Parent | on HN ]

Re:62% of code generated by GPT-4 contains API misuses by ldhough on Sunday August 27, 2023 @07:52PM (#37287883)

I've noticed that one of the most common failure patterns I get from GPT4 for code generation is that it incorrectly asserts something and then corrects itself in the same response.

ex: "This code `(some-fn 1 2)` does x because y. That is incorrect because abc"

I wondered if this has to do with common StackOverflow post formats.

[ Parent | on HN ]