Re:How to Run Llama 3 405B on Home Devices? Build AI Cluster
by atemerev on Monday July 29, 2024 @03:23PM
(#41102975)
Well, I live in an apartment in Switzerland, we have different standards.
[ Parent
| on HN ]
Re:How to Run Llama 3 405B on Home Devices? Build AI Cluster
by angoragoats on Monday July 29, 2024 @03:42PM
(#41103110)
Yeah, makes sense. I was just trying to point out that "you won't have enough elecricity to run it at home" is not true for many single-family homeowners.
I'm not familiar with the typical Swiss apartment, so I'm sorry if this is a stupid question, but do you not think that your apartment could sustain a ~15A load at 230V? That's how much you'd need for ten RTX 4090s at full power.
Re:How to Run Llama 3 405B on Home Devices? Build AI Cluster
by atemerev on Monday July 29, 2024 @05:02PM
(#41103773)
It is not just the video cards. CPUs, mainboards, cooling systems, losses/inefficiencies… E.g. 4x4090 Comino Grando workstation is rated for 2811W, which is OK. 8x will probably trip my circuit breaker already, unless I plug it into higher-power circuit for kitchen appliances.
Re:How to Run Llama 3 405B on Home Devices? Build AI Cluster
by angoragoats on Monday July 29, 2024 @05:57PM
(#41104179)
Yep, in the US you’d need to use at least two separate circuits as well. I’m just saying that there probably is sufficient electricity for a crazy 10+ GPU setup, in most single family houses at least.
Re:How to Run Llama 3 405B on Home Devices? Build AI Cluster
by BizarroLand on Tuesday July 30, 2024 @03:12PM
(#41113024)
If you're going that far out you might as well just tap into your breaker box and add a 40a 110v circuit. As long as you can run your cluster near the breaker box it would not be a huge expense to do that, and if like most homes your breaker is in the garage then that would make venting the excess heat out fairly simple.
Re:How to Run Llama 3 405B on Home Devices? Build AI Cluster
by angoragoats on Tuesday July 30, 2024 @05:51PM
(#41114490)
I don’t condone doing this under any circumstances without first consulting an electrician. Lots of things could go wrong; for example, 40A @ 120V is not a very common configuration, and most standard household receptacles are rated for only 20A at most. Consult an electrician, folks.
Re:How to Run Llama 3 405B on Home Devices? Build AI Cluster
by BizarroLand on Wednesday July 31, 2024 @01:04PM
(#41120986)
Thanks, I did mean to imply that you would be paying an electrician to do the work to code, but I did also overlook that someone who wants to run a 405b Llama ai at home might be more of a diy'er than the average person.
Re:How to Run Llama 3 405B on Home Devices? Build AI Cluster
by Kirby64 on Tuesday July 30, 2024 @07:39PM
(#41115168)
Or just run your PSU off 240V. Much more standard supply, and most switched mode supplies already support 240V. They’re typically more efficient, too.
Re:How to Run Llama 3 405B on Home Devices? Build AI Cluster
by angoragoats on Wednesday July 31, 2024 @04:06PM
(#41122925)
That works for sure, though it's less common in the US to have a 240v circuit available (with an appropriate receptacle) anywhere you might put a large server. Using two 120v circuits, you're more likely to be able to set it up with what you have now, without requiring any electrical work.
Re:How to Run Llama 3 405B on Home Devices? Build AI Cluster
by Kirby64 on Wednesday July 31, 2024 @04:12PM
(#41122963)
The comparison was a 120V 40A circuit. A 240V-20A socket would be more common than a 120V 50A socket. Both technically exist, but a 240V 20A outlet would be cheaper, safer, and more common. A 240V 20A outlet is actually somewhat common for larger window mounted AC units.
Re:How to Run Llama 3 405B on Home Devices? Build AI Cluster
by angoragoats on Wednesday July 31, 2024 @07:29PM
(#41124638)
> The comparison was a 120V 40A circuit.
Yes, which was a suggestion in reply to my comment mentioning the use of two standard 120v circuits, which I still maintain is the easiest and cheapest path for most people.