Mistral: Mixtral of Experts

Re:Mistral: Mixtral of Experts by jstummbillig on Monday December 11, 2023 @04:19AM (#38598938)

I am not sure what that means. How can they run GPT 3.5 any more or less?

[ Parent | on HN ]

Re:Mistral: Mixtral of Experts by simonvc on Monday December 11, 2023 @04:25AM (#38598977)

GPT-3.5 (unfinetuned) has been matched by many OW (open weights) models now, and with fine tuning to a specific task (coding, customer care, health advice etc) can exceed it.

It's still useful as a well known model to compare with, since it's the model the most people have experience with.

[ Parent | on HN ]

Re:Mistral: Mixtral of Experts by stsquad on Monday December 11, 2023 @04:54AM (#38599117)

I think they mean the assumed parameter size of GPT-4 is so large you couldn't run it on commodity hardware even if you could get hold of the model.

[ Parent | on HN ]

Re:Mistral: Mixtral of Experts by karmasimida on Monday December 11, 2023 @04:59AM (#38599136)

The rumored GPT4 model size is 1.8 trillion parameters

[ Parent | on HN ]