Building Unorthodox Deep Learning GPU Machines

Re:Building Unorthodox Deep Learning GPU Machines by verditelabs on Tuesday February 27, 2024 @10:48PM (#39533740)

I think I recognize the author of this from /r/localllama, where plenty of other people are building similar frankenstein rigs. This post only mentions Intel setups, but AMD Epyc Milan and Rome based rigs are also very viable alternatives. They're a bit more expensive, but much better perf/watt, and the incremental price increase after factoring in a lot of GPUs is fairly slim. With 7 PCIE bifurcators on a motherboard such as the AsRock ROMED8-2T and a 14 risers, you can get up to 14 GPUs at PCIE 4.0x8.

[ Parent | on HN ]

Re:Building Unorthodox Deep Learning GPU Machines by 3abiton on Wednesday February 28, 2024 @06:43AM (#39536639)

A bit sad hobbyist have to resort to such measures to get tinkering, not to mention the initial capital needed. We're all slaves to Nvidia's VRAM monopoly until AMD or Intel steps in and release a competitive alternative with beefy vrams.

[ Parent | on HN ]

Re:Building Unorthodox Deep Learning GPU Machines by loudmax on Wednesday February 28, 2024 @09:03AM (#39538038)

Beefy VRAM is a start, but Nvidia's real moat is CUDA. If PyTorch runs on AMD's ROCm, or the Intel equivalent, as well as it runs on CUDA, then we'll see some real competition here.

Chris Lattner's Mojo programming language may present an alternative here, but it's still closed source.

[ Parent | on HN ]

Re:Building Unorthodox Deep Learning GPU Machines by 3abiton on Thursday February 29, 2024 @04:02AM (#39547670)

How will Mojo solve what is essentially a GPU driver issue (ROCm vs cuda)? AMD's track at supporting GPU computing is very poor to be trusted so far.

[ Parent | on HN ]