Re:Coding with LLMs in the summer of 2025 (an update)
by jmb99 on Monday July 21, 2025 @12:53PM
(#44637427)
Yep, ~1kW as mentioned. Depending on your electrical rate, break even might be years down the line. And obviously the Mac Studios would perform substantially better.
Edit: And also, to get even half as much memory, you need to spend $10k. If you want to host actually-large LLMs (not quantized/distilled versions), you'll need to spend close to that much. Maybe you can get away with 256GB for now, but that won't even host full Deepseek now (and I don't know if 512GB either, with OS/etc overhead, and a large context window).
Edit: And also, to get even half as much memory, you need to spend $10k. If you want to host actually-large LLMs (not quantized/distilled versions), you'll need to spend close to that much. Maybe you can get away with 256GB for now, but that won't even host full Deepseek now (and I don't know if 512GB either, with OS/etc overhead, and a large context window).