Re:Self-Hosting Open Source LLMs: Cross Devices and Local De...
by xrd on Tuesday November 21, 2023 @08:37AM
(#38363414)
I really like the post that they mention (https://www.secondstate.io/articles/fast-llm-inference/). The reasons for avoiding python all resonate with me. I'm excited to play with WASI-NN (https://github.com/WebAssembly/wasi-nn) and that rust code is very readable to load up a GGUL model.
[ Parent
| on HN ]
Re:Self-Hosting Open Source LLMs: Cross Devices and Local De...
by euazOn on Tuesday November 21, 2023 @10:27AM
(#38364881)
Related discussion: https://news.ycombinator.com/item?id=38246668