Re:GPT-4 Details Leaked
by _nalply on Tuesday July 11, 2023 @11:21AM
(#36682252)
I remember to have read somewhere that GPT4 is not a single model but several models whose parameter counts are reported as a single sum. Perhaps quite doable but at lower speeds?
[ Parent
| on HN ]
Re:GPT-4 Details Leaked
by kirill5pol on Tuesday July 11, 2023 @12:11PM
(#36683069)
The article linked here talks about GPT4 being a mixture of experts, which is exactly what you’re describing