News for hackers. Stuff that matters.

A comment on Show HN: LlamaGPT – Self-hosted, offline, private AI chatbot, powered by Llama 2. Parent comment

Re:Show HN: LlamaGPT – Self-hosted, offline, private AI chat... by mayankchhabra on Wednesday August 16, 2023 @06:47PM (#37154903)
Not sure how powerful their NAS is, but on Umbrel Home (which has an N5105 CPU), it's pretty useable with ~3 tokens generated per second.
[ Parent | on HN ]

There are no comments on this comment yet.