News for hackers. Stuff that matters.
Re:Meta Llama 3 by qeternity on Thursday April 18, 2024 @06:24PM (#40081390)
I'm pretty sure he suggested it was a 16 way 110 MoE
Re:Meta Llama 3 by brandall10 on Friday April 19, 2024 @11:45AM (#40088262)
The exact quote: "Sam Altman won’t tell you that GPT 4 has 220 billion parameters and is a 16 way mixture model with eight sets of weights."