News for hackers. Stuff that matters.

A comment on Phind-70B: Closing the code quality gap with GPT-4 Turbo while running 4x faster. Parent comment

Re:Phind-70B: Closing the code quality gap with GPT-4 Turbo ... by benxh on Friday February 23, 2024 @01:05PM (#39484032)
If GPT4 is 220B/8 experts, that would be in-line with 3.5 Turbo being a 20B model, and GPT4 being a 55B activation out of a total 220B parameters.

It is ultimately all speculation, until Deepseek releases their own 145B MoE model, and then we can compare the activations/results

[ Parent | on HN ]
Re:Phind-70B: Closing the code quality gap with GPT-4 Turbo ... by devit on Friday February 23, 2024 @03:34PM (#39485762)
I think the conjecture is that each expert of GPT-4 has 220B parameters, for a total of 1.76T parameters.
[ Parent | on HN ]