Re:LLaMA Now Goes Faster on CPUs
by samus on Monday April 01, 2024 @02:24AM
(#39891364)
The abstractions are relatively brittle. If you don't have a powerful GPU, you will be forced to consider how to split the model between CPU and GPU, how much context size you need, whether to quantize the model, and the tradeoffs implied by these things. To understand these, you have to develop a basic model how an LLM works.
[ Parent
| on HN ]
There are no comments on this comment yet.