Re:Meta Llama 3
by cjbprime on Thursday April 18, 2024 @01:41PM
(#40078719)
The rumor is that it's a mixture of experts model, which can't be compared directly on parameter count like this because most weights are unused by most inference passes. (So, it's possible that 400B non-MoE is the same approximate "strength" as 1.8T MoE in general.)
[ Parent
| on HN ]
There are no comments on this comment yet.