Re:I Run LLMs Locally
by philjohn on Sunday December 29, 2024 @04:48PM
(#42543608)
Not entirely.
TRAINING an LLM requires a lot of compute. Running inference on a pre-trained LLM is less computationally expensive, to the point where you can run LLAMA (cost $$$ to train on Meta's GPU cluster) with CPU-based inference.
TRAINING an LLM requires a lot of compute. Running inference on a pre-trained LLM is less computationally expensive, to the point where you can run LLAMA (cost $$$ to train on Meta's GPU cluster) with CPU-based inference.