News for hackers. Stuff that matters.
Re:Meta Llama 3 by Davidzheng on Thursday April 18, 2024 @01:34PM (#40078662)
it's in the official post the comparison with Mixtral 8x22B
Re:Meta Llama 3 by cjbprime on Thursday April 18, 2024 @01:37PM (#40078687)
Where? I only see comparisons to Mistral 7B and Mistral Medium, which are totally different models.
Re:Meta Llama 3 by gs17 on Thursday April 18, 2024 @01:55PM (#40078874)
https://ai.meta.com/blog/meta-llama-3/ has it about a third of the way down. It's a little bit better on every benchmark than Mixtral 8x22B (according to Meta).
Re:Meta Llama 3 by cjbprime on Thursday April 18, 2024 @10:59PM (#40083061)
Oh cool! But at the cost of twice the VRAM and only having 1/8th of the context, I suppose?
Re:Meta Llama 3 by modeless on Saturday April 20, 2024 @02:49AM (#40095385)
Llama 3 70B takes half the VRAM as Mixtral 8x22B. But it does need almost twice the FLOPS/bandwidth. Yes, Llama's context is smaller although that should be fixable in the near future. Another thing is that Llama is English-focused while Mixtral is more multilingual.