Re:Llama.cpp guide – Running LLMs locally on any hardware, f...
by anon373839 on Saturday November 30, 2024 @05:33PM
(#42284528)
Two points: 1) I actually think that smaller models are substantially similar to frontier models. Of course the latter are more capable, but they’re more similar than different (which I think the ELO scores on lmarena.ai suggests).
2) You can run much larger models on Apple Silicon with surprisingly decent speed.
2) You can run much larger models on Apple Silicon with surprisingly decent speed.