News for hackers. Stuff that matters.

A comment on The Coming of Local LLMs. Parent comment

Re:The Coming of Local LLMs by kbrkbr on Tuesday April 11, 2023 @12:39PM (#35527587)
I started with ggerganov’s llama.cpp GitHub repo, and went from there. But then again, I know some programming, statistics and machine learning, so it may not be for you, I cannot judge that.

Models can be found on huggingface.co, and I’d start with eachadea/ggml-vicuna-13b-4bit, but it needs 10G of cpu-ram. It is very friendly to any prompt though.

I read on my way (on reddit, when I recall correctly), that there must be some really good intro videos on YouTube.

[ Parent | on HN ]
Re:The Coming of Local LLMs by boppo1 on Wednesday April 12, 2023 @12:42AM (#35535101)
additional point of reference I don't know shit about ML or stats, and am a very weak programmer. I just know how to install programs on linux and use CLI in a basic fashion. I have had no problem getting llama.cpp going by following ggerganov's readme.
[ Parent | on HN ]