News for hackers. Stuff that matters.
Re:All of ChatGPT's System Prompts by andmonad on Saturday October 14, 2023 @07:14AM (#37879626)
But how are system messages given to GPT, are there any other lower level prompts? This may be outdated but last I remember ChatGPT is just GPT with a prompt like

  The following is a chat between an AI and a user:

  - AI: How can I help?
  - User: ...
At least that's how I simulated chats on the OpenAI playground before ChatGPT.

Is this done differently now, or if not I wonder if anyone has been able to guess what that prompt says and how the system message gets inserted.

Re:All of ChatGPT's System Prompts by Tiberium on Saturday October 14, 2023 @07:40AM (#37879734)
There are no lower-level prompts, the messages are formatted using ChatML [1] which has special tokens to denote messages in the chat context that the model was trained on, so it can clearly differentiate between different messages.

[1] https://github.com/openai/openai-python/blob/main/chatml.md

Re:All of ChatGPT's System Prompts by spdustin on Saturday October 14, 2023 @11:14AM (#37881121)
I have suspicions that there is middleware somewhere for the ChatGPT interface to the chat models underneath; something to enclose the normal prompt, to update weights, or to manipulate logits.

Just last night I began seeing a behavior that I could formerly reproduce 100% of the time: asking it to critically evaluate my instructions, explaining why it didn’t follow them, and suggest rewrites. Since the beginning of ChatGPT itself, it would reliably answer that every time. As of last night, it flat out refused to, assuring me of its sincere apologies and confidently stating it’ll follow my instructions better from now on.

Re:All of ChatGPT's System Prompts by simbolit on Saturday October 14, 2023 @07:37AM (#37879724)
If I understand correctly, the older method you describe has been replaced by exposing a GPT model to some further training (as opposed to "pre-training") with successful conversations. I think this premiered with the InstructGPT paper: https://arxiv.org/pdf/2203.02155.pdf