News for hackers. Stuff that matters.

A comment on The Coming of Local LLMs. Parent comment

Re:The Coming of Local LLMs by travisjungroth on Tuesday April 11, 2023 @12:08PM (#35527170)
It’s in the near past. https://github.com/ggerganov/llama.cpp
[ Parent | on HN ]
Re:The Coming of Local LLMs by abetlen on Tuesday April 11, 2023 @12:17PM (#35527311)
Also worth checking out https://github.com/saharNooby/rwkv.cpp which is based on Georgi's library and offers support for the RWKV family of models which are Apache-2.0 licensed.
[ Parent | on HN ]
Re:The Coming of Local LLMs by BrutalCoding on Tuesday April 11, 2023 @01:27PM (#35528273)
I’ve got some of their smaller Raven models running locally on my M1 (only 16GB of RAM).

I’m also in the middle of making it user friendly to run these models on all platforms (built with Flutter). First MacOS release will be out before this weekend: https://github.com/BrutalCoding/shady.ai

[ Parent | on HN ]