Re:It looks like GPT-4-32k is rolling out
by reaperman on Saturday May 06, 2023 @04:49PM
(#35845240)
Ah this explains a lot. I couldn't understand why I couldn't get close to the ~12 pages that everyone was saying 8,000 tokens implied.
Re:It looks like GPT-4-32k is rolling out
by tempaccount420 on Saturday May 06, 2023 @04:57PM
(#35845301)
As far as I know it's not documented anywhere and there is no way to ask the team at ChatGPT questions. I sent them an email about it a few days after GPT-4 release and still haven't received a reply.
Another thing that annoys me is how most updates don't get a changelog entry. For whatever reason, they keep little secrets like that.
Re:It looks like GPT-4-32k is rolling out
by jiggawatts on Sunday May 07, 2023 @12:45AM
(#35848016)
Their PR is terrible and I get the impression that they wish their own users would “just go away”.
Every time I see a company act like this, more responsive and truly open competition eventually eats their lunch.
The raw chat log has the system message on top, plus "user:" and "assistant:" for each message, and im_start/im_end tokens to separate messages, hence why the visible chat context is slightly under 4k.
I'm pretty sure they're using a 4k GPT-4 model for ChatGPT Plus, even though they only announced 8k and 32k... It can't handle more than 4k of tokens (actually a little below that, starts ignoring your last few sentences if you get close). If you check developer tools, the API to a /models endpoint says the limit for GPT-4 is 4096. It's very unfortunate.