News for hackers. Stuff that matters.
Re:John Carmack: From a DM, just in case anyone else needs t... by execveat on Sunday March 19, 2023 @06:41AM (#35217720)
Yeah, but now ask it to write a program that uses this API and then let it debug problems which arise from the swagger spec (or the backend) having bugs. I don't think LLMs have any way of recognizing and dealing with bad input data. That is I don't think they can recognize when something that is supposed to work in a particular way doesn't and fixing it is completely out of your reach, but you still need to get things working (by introducing workarounds).
Re:John Carmack: From a DM, just in case anyone else needs t... by sebzim4500 on Sunday March 19, 2023 @10:02AM (#35218997)
Have you tried it? If you copy the errors back into the chat I could imagine it working quite well. Certainly you can give it contradictory instructions and it makes a decent effort at following them.
Re:John Carmack: From a DM, just in case anyone else needs t... by execveat on Sunday March 19, 2023 @11:14AM (#35219792)
Yes, I'm subscribed to poe.com and am playing with all public models. They all suck at debugging issues with no known answers (I'm talking about typical problems every software developer, DevOps or infosec person solves every day).

You need a real ability to reason and preserve context beyond inherent context window somehow (we humans do it by keeping notes, writing emails, and filing JIRA tickets). So while this doesn't require full AGI and some form of AI might be able to do it this century, it won't be LLMs.

Re:John Carmack: From a DM, just in case anyone else needs t... by ilaksh on Sunday March 19, 2023 @11:39AM (#35220076)
If you think that the average public LLM is equivalent to ChatGPT or GPT-4 then you are completely mistaken. By a factor of say 500-10000%.
Re:John Carmack: From a DM, just in case anyone else needs t... by execveat on Monday March 20, 2023 @07:41AM (#35230138)
poe.com is a web interface (by Quora) to multiple LLMs. Right now it's ChatGPT, GPT-4, Claude, Claude+ as well as Sage and Dragonfly.