News for hackers. Stuff that matters.
Re:Cost Per 1M tokens Of Self Hosting Llama-3 by christina97 on Friday June 14, 2024 @01:41PM (#40682956)
If you integrate with existing tooling, it won’t do this optimization. Unless of course you really go crazy with your setup.
Re:Cost Per 1M tokens Of Self Hosting Llama-3 by moffkalast on Friday June 14, 2024 @05:36PM (#40685373)
Setting one launch flag on llama.cpp server hardly qualifies as going crazy with one's setup.