News for hackers. Stuff that matters.

A comment on Meta Llama 3. Parent comment

Re:Meta Llama 3 by schleck8 on Thursday April 18, 2024 @01:29PM (#40078615)
You absolutely can since it has a size advantage either way. MoE means the expert model performs better BECAUSE of the overall model size.
[ Parent | on HN ]
Re:Meta Llama 3 by cjbprime on Thursday April 18, 2024 @01:39PM (#40078706)
Fair enough, although it means we don't know whether a 1.8T MoE GPT-4 will have a "size advantage" over Llama 3 400B.
[ Parent | on HN ]