News for hackers. Stuff that matters.

A comment on Apple M3 Ultra. Parent comment

Re:Apple M3 Ultra by wmf on Wednesday March 05, 2025 @04:12PM (#43272492)
My understanding is that if you can store 1/Nth of the weights in RAM on each of the N nodes then there's no need to send the weights over the network.
[ Parent | on HN ]
Re:Apple M3 Ultra by unsatchmo on Thursday March 06, 2025 @11:37AM (#43282138)
You're correct about the weights: each machine could in fact store all of the weights. However I think you still have to transfer the activations and the KV-Cache while performing inference.
[ Parent | on HN ]