News for hackers. Stuff that matters.
Re:Meta Llama 3 by staticman2 on Monday April 22, 2024 @09:08AM (#40114000)
"For example 8x7B Mixtral only executes 13B parameters per token, but it behaves comparable to 34B and 70B models"

Are you sure about that? I'm pretty sure Miqu (the leaked Mistral 70b model) is generally thought to be smarter than Mixtral 8x7b.

There are no comments on this comment yet.