Re:I Run LLMs Locally
by ein0p on Thursday January 09, 2025 @02:13PM
(#42648890)
I'm running 123B parameter Mistral Large with no issues. Larger models will run, too, but slowly. I wish Ollama had support for speculative decoding.
[ Parent
| on HN ]