GPT-4.5 or GPT-5 being tested on LMSYS?

Re:GPT-4.5 or GPT-5 being tested on LMSYS? by hehdhdjehehegwv on Monday April 29, 2024 @09:25PM (#40206292)

Funny thing is I’m still in love with Mistral 7B as it absolutely shreds on a nice GPU. For simple tasks it’s totally sufficient.

[ Parent | on HN ]

Re:GPT-4.5 or GPT-5 being tested on LMSYS? by qeternity on Tuesday April 30, 2024 @03:56AM (#40208337)

Llama3 8B is for all intents and purposes just as fast.

[ Parent | on HN ]

Re:GPT-4.5 or GPT-5 being tested on LMSYS? by minosu on Tuesday April 30, 2024 @10:42AM (#40211453)

Mistral 7b inferences about 18% faster for me as a 4bit quantized version on an A100. Thats definitely relevant when running anything but chatbots.

[ Parent | on HN ]

Re:GPT-4.5 or GPT-5 being tested on LMSYS? by tmostak on Tuesday April 30, 2024 @04:52PM (#40216200)

Are you measuring tokens/sec or words per second?

The difference matters as generally in my experience, Llama 3, by virtue of its giant vocabulary, generally tokenizes text with 20-25% less tokens than something like Mistral. So even if its 18% slower in terms of tokens/second, it may, depending on the text content, actually output a given body of text faster.

[ Parent | on HN ]