Re:Show HN: LlamaGPT – Self-hosted, offline, private AI chat...
by ryanSrich on Wednesday August 16, 2023 @04:28PM
(#37153319)
Interesting. I might try to get this to work on my NAS.
[ Parent
| on HN ]
Re:Show HN: LlamaGPT – Self-hosted, offline, private AI chat...
by reneberlin on Wednesday August 16, 2023 @06:42PM
(#37154861)
Good luck! The token/sec will be under your expectations or it will overheat. You really shouldn't play games with your data-storage. You could try it with an old laptop to see how bad it performs. Ruining your NAS for this is a bit over the top to show, that "it worked somehow".
But i don't know, maybe your NAS has a powerful processor and is tuned to the max and you have redundancy and don't care to loose a NAS?
Or this was just a joke and i fell for it! ;)
Re:Show HN: LlamaGPT – Self-hosted, offline, private AI chat...
by mayankchhabra on Wednesday August 16, 2023 @06:47PM
(#37154903)
Not sure how powerful their NAS is, but on Umbrel Home (which has an N5105 CPU), it's pretty useable with ~3 tokens generated per second.