Re:Serving AI from the Basement – 192GB of VRAM Setup
by mattnewton on Monday September 09, 2024 @05:43PM
(#41494320)
Thanks! Yeah the 4090’s are very thirsty if you let them be, I haven’t played enough with throttling their voltage and how that affects perf. Looking forward to your articles.
So, as mentioned on the article, I actually have installed (2) 30amp 240v breakers dedicated entirely for this setup (and the next one in case I decide to expand to 16x GPUs over 2 nodes lol). Each breaker is supposed to power up to 6000w at ease. I also installed a specific kind of power outlet that can handle that kind of current, and I have done some extreme research into PDUs. I plan on posting about all of that in this series (part 3 according to my current tentative plans) so stay tuned and maybe bookmark the website/add the RSS feed to your digest/or follow me on any of the socials if this is something that you wanna nail down without spending a month on research like me :'D