News for hackers. Stuff that matters.
Re:Cost Per 1M tokens Of Self Hosting Llama-3 by wkat4242 on Saturday June 15, 2024 @10:05PM (#40694168)
No, the ollama default quantisation is 4 bit
Re:Cost Per 1M tokens Of Self Hosting Llama-3 by brrrrrm on Sunday June 16, 2024 @07:44AM (#40696343)
I meant 8b -> 8billion rather than 70b
Re:Cost Per 1M tokens Of Self Hosting Llama-3 by wkat4242 on Sunday June 16, 2024 @01:37PM (#40698728)
Ah sorry!