News for hackers. Stuff that matters.

A comment on Phi-3 Technical Report. Parent comment

Re:Phi-3 Technical Report by oersted on Monday April 22, 2024 @11:32PM (#40128170)
In the English category: GPT-4-0314 (ELO 1166), Llama 3 8B Instruct (ELO 1161), Mistral-Large-2402 (ELO 1151), GPT-4-0613 (ELO 1148).

You are right, I toned down the language, I got a bit overexcited, and I missed the difference in the versions of GPT-4. And LMSYS is a subjective benchmark for what users prefer, which I'm sure has weird inherent biases.

It's just that any signal of an 3.8B model being anywhere in the vicinity of GPT-4 is huge.

[ Parent | on HN ]
Re:Phi-3 Technical Report by moralestapia on Monday April 22, 2024 @11:39PM (#40128210)
Yeah, GPT3.5, in a phone, at ~1,000 tokens/sec ... nice!
[ Parent | on HN ]
Re:Phi-3 Technical Report by mlyle on Tuesday April 23, 2024 @12:05AM (#40128376)
> at ~1,000 tokens/sec

12 tokens per second.

[ Parent | on HN ]
Re:Phi-3 Technical Report by moralestapia on Tuesday April 23, 2024 @07:53AM (#40130855)
Whoops, made the same mistake as @ignoramous :P
[ Parent | on HN ]