News for hackers. Stuff that matters.
Re:I Run LLMs Locally by fnqi8ckfek on Sunday December 29, 2024 @05:21PM (#42543925)
I know nothing about gpus. Should I be assuming that when people say "ram" in the context of gpus they always mean vram?
Re:I Run LLMs Locally by elorant on Sunday December 29, 2024 @06:55PM (#42544770)
Not always, because system RAM also has to be equally adequate, but mostly yes, it's about the total VRAM of the GPU(s).
Re:I Run LLMs Locally by layer8 on Sunday December 29, 2024 @05:27PM (#42543972)
“GPU with xx RAM” means VRAM, yes.