Re:Mixtral 8x22B
by moffkalast on Wednesday April 17, 2024 @01:14PM
(#40067538)
How does that make any sense on a decoder-only architecture?
[ Parent
| on HN ]
It's not about the model. The model can output more - it's about the API.
A better phrasing would be that they don't allow you to output more than 4k tokens per message.
Same with Anthropic and Claude, sadly.