News for hackers. Stuff that matters.
Re:I Run LLMs Locally by Abishek_Muthian on Monday December 30, 2024 @04:16AM (#42547800)
OP here, I almost got a decked out Mac studio before I returned it for a Asus ROG as the native Linux support, upgradability & CUDA support is much more important to me.

Meagre VRAM in these Nvidia consumer GPUs is indeed painful but with increasing performance of smaller LLMs & fine tuned models I don't think 12GB, 14GB, 16GB Nvidia GPUs offering much better performance over a Mac can be easily dismissed.

There are no comments on this comment yet.