Re:Don Knuth Plays with ChatGPT
by sltkr on Saturday May 20, 2023 @05:40PM
(#36015867)
It's jarring because you can give it an instruction like "I want you to talk like a pirate from now on" or "Please end every reply with 'beep boop'" and it will happily oblige and continue to do what was asked continually. So it's strange that being apologetic is apparently so ingrained in its training that it cannot stop apologizing even when you specifically requested that it refrained from doing so.
[ Parent
| on HN ]
I think it's more strange that the coincidence of some imperative prompts generating texts that looks like it's answering to it makes intelligent people think it's reasoning about the instructions and responding meaningfully to them instead of, you know, generating text as it was programmed to do.
Re:Don Knuth Plays with ChatGPT
by HarHarVeryFunny on Sunday May 21, 2023 @06:48PM
(#36025338)
It treats the system prompt specially - it's more than just a prefix to the conversation. OpenAI have said they put a lot of work it making it follow it as closely as possible (but no details of how they did it).