News for hackers. Stuff that matters.
Re:Programming with Natural Language Is Going to Work (2010) by Turskarama on Wednesday May 24, 2023 @04:01AM (#36055273)
Yeah, it does great on little toy examples.

What I would like to do is feed in my entire 50k line program and get something out.

Re:Programming with Natural Language Is Going to Work (2010) by moffkalast on Wednesday May 24, 2023 @06:12AM (#36055969)
I really wonder how Claude 100k does on larger workspaces, has anyone tried that? (I don't feel like paying another $20 to Anthropic too) Allegedly it's only marginally better than 3.5-turbo on average so it'll probably spit out nonsensical code but maybe the huge context can help.
Re:Programming with Natural Language Is Going to Work (2010) by cookieperson on Wednesday May 24, 2023 @07:19AM (#36056432)
Or you can just write the code yourself instead of praying something else can do your job for you better than you can...
Re:Programming with Natural Language Is Going to Work (2010) by moffkalast on Wednesday May 24, 2023 @09:06AM (#36057464)
But that would require me to not be a lazy ass, which we all know is impossible.

Also writing code, in 2023? With your hands? Pshh, that's so 2010s.

Re:Programming with Natural Language Is Going to Work (2010) by byby on Wednesday May 24, 2023 @11:12AM (#36059259)
So I said it's like 50 percent of the way there implying that it gets things right at a rate of 50 percent. That's a fuzzy estimation as well, obviously so don't get pedantic on me with that number.

When you ask for large output or give it large input you are increasing the sample size. Which means more likely that part of the answer are wrong. That's it. Simple statistic that are inline with my initial point. With AI we are roughly half way there at producing answers.

If you keep the answers and questions short you will have a much higher probability of being correct.

So that 50k line program? My claim is roughly 25k of those lines are usable. But that's a fuzzy claim because I LLMs can do much better than 25k. Maybe 75% is more realistic but I'll leave it at 50% so there's a lower bar for the nay sayers to attack.