News for hackers. Stuff that matters.
Re:Making o1, o3, and Sonnet 3.7 Hallucinate for Everyone by xmprt on Saturday March 01, 2025 @05:13PM (#43224277)
In my experience using LLMs, the 90% is less about buggy code and more about just ignoring 10% of the features that you require. So it will write code that's mostly correct in 100-1000 lines of code (not buggy) but then no matter how hard you try, it won't get the remaining 10% right and in the process, it will mess up parts of the 90% that was already working or end up writing another 1000 lines of undecipherable code to get 97% there but still never 100% unless you're building something that's not that unique.
Re:Making o1, o3, and Sonnet 3.7 Hallucinate for Everyone by andix on Saturday March 01, 2025 @05:43PM (#43224625)
Exactly my experience. It's always missing something. And the generated code often can't be extended to fulfil those missing aspects.