News for hackers. Stuff that matters.
Re:Self-Hosting Open Source LLMs: Cross Devices and Local De... by xrd on Tuesday November 21, 2023 @08:37AM (#38363414)
I really like the post that they mention (https://www.secondstate.io/articles/fast-llm-inference/). The reasons for avoiding python all resonate with me. I'm excited to play with WASI-NN (https://github.com/WebAssembly/wasi-nn) and that rust code is very readable to load up a GGUL model.
Re:Self-Hosting Open Source LLMs: Cross Devices and Local De... by euazOn on Tuesday November 21, 2023 @10:27AM (#38364881)