News for hackers. Stuff that matters.

A comment on Meta Llama 3. Parent comment

Re:Meta Llama 3 by schleck8 on Thursday April 18, 2024 @12:31PM (#40077966)
> Neglected to include comparisons against GPT-4-Turbo or Claude Opus, so I guess it's far from being a frontier model

Yeah, almost like comparing a 70b model with a 1.8 trillion parameter model doesn't make any sense when you have a 400b model pending release.

[ Parent | on HN ]
Re:Meta Llama 3 by cjbprime on Thursday April 18, 2024 @01:18PM (#40078511)
(You can't compare parameter count with a mixture of experts model, which is what the 1.8T rumor says that GPT-4 is.)
[ Parent | on HN ]
Re:Meta Llama 3 by schleck8 on Thursday April 18, 2024 @01:29PM (#40078615)
You absolutely can since it has a size advantage either way. MoE means the expert model performs better BECAUSE of the overall model size.
[ Parent | on HN ]
Re:Meta Llama 3 by cjbprime on Thursday April 18, 2024 @01:39PM (#40078706)
Fair enough, although it means we don't know whether a 1.8T MoE GPT-4 will have a "size advantage" over Llama 3 400B.
[ Parent | on HN ]