News for hackers. Stuff that matters.
Re:Apple M3 Ultra by aurareturn on Wednesday March 05, 2025 @11:26AM (#43268720)
It's good enough to run whatever local model you want. 2x 80core GPU is no joke. Linking them together gives it effectively 1.6 TB/s of bandwidth. 1TB of total memory.

You can run the full Deepseek 671b q8 model at 40 tokens/s. Q4 model at 80 tokens/s. 37B active params at a time because R1 is MoE.

There are no comments on this comment yet.