News for hackers. Stuff that matters.
Re:Mistral NeMo by voiper1 on Thursday July 18, 2024 @11:19AM (#40996391)
Oddly, they are only charging slightly more for their hosted version:

open-mistral-7b is 25c/m tokens open-mistral-nemo-2407 is 30c/m tokens

https://mistral.ai/technology/#pricing

Re:Mistral NeMo by Palmik on Thursday July 18, 2024 @12:11PM (#40997083)
They specifically call out fp8 aware training and TensoRT LLM is really good (efficient) with fp8 inference on H100 and other hopper cards. It's possible that they run the 7b natively in fp16 as smaller models suffer more from even "modest" quantization like this.
Re:Mistral NeMo by dannyw on Thursday July 18, 2024 @11:41AM (#40996718)
Possibly a NVIDIA subsidy. You run NEMO models, you get cheaper GPUs.