Re:OpenAI Employee: GPT-4 hasn't gotten worse since March
by gwern on Thursday June 01, 2023 @07:24PM
(#36158822)
This should always be the first explanation when you're speculating about something involving ChatGPT or Bing Sydney, if you aren't directly using the API and comparing temp=0 responses to identical canned prompts. There are lots of smaller models & filters and prompts involved, all of which are much easier to update than the behemoth underneath, and which do get updated frequently without notice. People jump the gun constantly on assuming it's the biggest model which changed.
[ Parent
| on HN ]
Re:OpenAI Employee: GPT-4 hasn't gotten worse since March
by robbintt on Sunday June 04, 2023 @10:35AM
(#36185906)
What is your hypothesis for the layout/diagram of models feeding the gpt-4 output for chatgpt/gpt-4 vs api/gpt-4?
Re:OpenAI Employee: GPT-4 hasn't gotten worse since March
by gwern on Sunday June 04, 2023 @12:21PM
(#36186862)
Components which are likely there and which could be affecting quality: the invisible-to-the-user 'system prompt' (probably with few-shot examples), retrieval from history (not necessarily exclusively your own, as a way to augment few-shot), cascaded models with a very cheap 'turbo' model to try to answer first, possibly regular finetunes of the main chat model (note that OP only specifies the API model hasn't changed, but the live chat models seem to change frequently), and a post-generation filtering model trying to reject offensive outputs.
We know MS Bing Sydney is doing at least 3 of those (prompt, cascade with Megatron, and finetuned rejection classifier for post-output filtering) on top of its GPT-4-finetune, so it's not a stretch to figure that OA is doing similar things.