News for hackers. Stuff that matters.
Re:A brief history of LLaMA models by int_19h on Saturday April 29, 2023 @06:55PM (#35758091)
The GPU isn't actually used by llama.cpp. What makes it that much faster is that the workload, either on CPU or on GPU, is very memory-intensive, so it benefits greatly from fast RAM. And Apple is using DDR5 running at very high clock speeds for this shared memory stuff.

It's still noticeably slower than GPU, though.

There are no comments on this comment yet.