Comparing to the numbers here https://www.anthropic.com/news/claude-3-family the ones of Llama 400B seem slightly lower, but of course it's just a checkpoint that they benchmarked and they are still training further.
Indeed. But if GPT-4 is actually 1.76T as rumored, an open-weight 400B is quite the achievement even if it's only just competitive.
The rumor is that it's a mixture of experts model, which can't be compared directly on parameter count like this because most weights are unused by most inference passes. (So, it's possible that 400B non-MoE is the same approximate "strength" as 1.8T MoE in general.)
Re:Meta Llama 3
by dmarchand90 on Thursday April 18, 2024 @12:25PM
(#40077882)
Where does it say much better than gpt4 for the 400B model?
Re:Meta Llama 3
by Davidzheng on Thursday April 18, 2024 @01:10PM
(#40078437)
It doesn't ....
Re:Meta Llama 3
by jasonjmcghee on Thursday April 18, 2024 @12:36PM
(#40078031)
It absolutely does not say that. It in fact provides benchmarks that show it under performing them.
Not great to blindly trust benchmarks, but there are no claims it will outperform GPT-4 or Opus.
It was a checkpoint, so it's POSSIBLE it COULD outperform.
Re:Meta Llama 3
by SV_BubbleTime on Thursday April 18, 2024 @12:43PM
(#40078118)
Is it decentralized? You can run it multiple places I guess, but it’s only available from one place.
And it’s not open source.
Re:Meta Llama 3
by 12345hn6789 on Thursday April 18, 2024 @01:06PM
(#40078402)
It's not open source or decentralized.
that's very exciting. are you quoting same benchmark comparisons?
Looks like there's a 400B version coming up that will be much better than GPT-4 and Claude Opus too. Decentralization and OSS for the win!