News for hackers. Stuff that matters.
Re:Mixtral 8x22B by pseudosavant on Wednesday April 17, 2024 @12:23PM (#40066865)
FWIW, the 128k context window for GPT-4 is only for input. I believe the output content is still only 4k.
Re:Mixtral 8x22B by moffkalast on Wednesday April 17, 2024 @01:14PM (#40067538)
How does that make any sense on a decoder-only architecture?
Re:Mixtral 8x22B by grey8 on Wednesday April 17, 2024 @02:38PM (#40068552)
It's not about the model. The model can output more - it's about the API.

A better phrasing would be that they don't allow you to output more than 4k tokens per message.

Same with Anthropic and Claude, sadly.

Re:Mixtral 8x22B by ljhskyso on Thursday April 18, 2024 @02:49AM (#40073613)
you can always put the unfinished output as the input to continue forever until reaching the full 128k context window