News for hackers. Stuff that matters.
Re:Ask HN: Will it be a viable business model to offer llama... by fbnbr on Tuesday September 26, 2023 @11:33PM (#37669678)
Why would you need it as cpp? Do you mean just llama or specifically a version small enough running on edge?
Re:Ask HN: Will it be a viable business model to offer llama... by throw03172019 on Wednesday September 27, 2023 @02:16AM (#37670841)
Do you mean just a non-fine tuned chat bot based on llama 70b? Probably not.
Re:Ask HN: Will it be a viable business model to offer llama... by quickthrower2 on Wednesday September 27, 2023 @06:56AM (#37672890)
I suspect no.
Re:Ask HN: Will it be a viable business model to offer llama... by version_five on Wednesday September 27, 2023 @12:06PM (#37676771)
Short answer, possibly, especially if it was say part of a rag system or some other architecture like that. There is room for more. Nothing particularly special about llama.cpp as the llm back end though. It's optimized for running on lower-end hardware which matters less if you're serving models as a service. But it has many strengths.

Llama.cpp / ggml is the open core of ggml.ai founded by GG and funded by the guy from github, so they have some monetization plan for it.