News for hackers. Stuff that matters.

A comment on GPT4 is 8 x 220B params = 1.7T params. Parent comment

Re:GPT4 is 8 x 220B params = 1.7T params by curiousgal on Wednesday June 21, 2023 @05:31AM (#36416346)
What exactly is "it"? What is being discussed here? The size of the model? Who cares? People will make the same API call to get the results. Everyone is acting like a glorified AI expert now smh
[ Parent | on HN ]
Re:GPT4 is 8 x 220B params = 1.7T params by HarHarVeryFunny on Wednesday June 21, 2023 @06:52AM (#36416896)
The rumor here is that GPT-4 isn't one giant model, but instead a "mixture of experts" combination of 8 smaller GPT-3 sized models with different training sets.

This has implications for what may be possible on consumer hardware.

[ Parent | on HN ]