News for hackers. Stuff that matters.

A comment on Apple M3 Ultra. Parent comment

Re:Apple M3 Ultra by aurareturn on Wednesday March 05, 2025 @11:23AM (#43268684) 

  what's the point of 512GB RAM for LLMs on this Mac Studio if the speed is painfully slow?
You can fit the entire Deepseek 671B q4 into this computer and get 41 tokens/s because it's an MoE model.
[ Parent | on HN ]
Re:Apple M3 Ultra by KingOfCoders on Wednesday March 05, 2025 @02:09PM (#43270842)
Your comments went from

"40 tokens/s by my calculations"

to

"40 tokens/s"

to

"41 tokens/s"

Is there a dice involved in "your calculations?"

[ Parent | on HN ]
Re:Apple M3 Ultra by aurareturn on Wednesday March 05, 2025 @08:59PM (#43275402)
41 was when I learned it has a little over 800B/s.

Doesn't matter. All theorized because no one has publicly tested one.

[ Parent | on HN ]