News for hackers. Stuff that matters.

A comment on How to Run Llama 3 405B on Home Devices? Build AI Cluster. Parent comment

Re:How to Run Llama 3 405B on Home Devices? Build AI Cluster by malux85 on Monday July 29, 2024 @04:48AM (#41098915)
Surely not with GPUs? Could you direct me to them please?
[ Parent | on HN ]
Re:How to Run Llama 3 405B on Home Devices? Build AI Cluster by RealStickman_ on Monday July 29, 2024 @12:19PM (#41101567)
This project only supports CPU inference at the moment, so no, no GPUs. I just estimated what some 256GB or RAM might cost monthly on the low end.
[ Parent | on HN ]