Re:John Carmack: From a DM, just in case anyone else needs t...
by execveat on Sunday March 19, 2023 @06:41AM
(#35217720)
Yeah, but now ask it to write a program that uses this API and then let it debug problems which arise from the swagger spec (or the backend) having bugs. I don't think LLMs have any way of recognizing and dealing with bad input data. That is I don't think they can recognize when something that is supposed to work in a particular way doesn't and fixing it is completely out of your reach, but you still need to get things working (by introducing workarounds).
[ Parent
| on HN ]
Re:John Carmack: From a DM, just in case anyone else needs t...
by sebzim4500 on Sunday March 19, 2023 @10:02AM
(#35218997)
Have you tried it? If you copy the errors back into the chat I could imagine it working quite well. Certainly you can give it contradictory instructions and it makes a decent effort at following them.
Re:John Carmack: From a DM, just in case anyone else needs t...
by execveat on Sunday March 19, 2023 @11:14AM
(#35219792)
Yes, I'm subscribed to poe.com and am playing with all public models. They all suck at debugging issues with no known answers (I'm talking about typical problems every software developer, DevOps or infosec person solves every day).
You need a real ability to reason and preserve context beyond inherent context window somehow (we humans do it by keeping notes, writing emails, and filing JIRA tickets). So while this doesn't require full AGI and some form of AI might be able to do it this century, it won't be LLMs.