News for hackers. Stuff that matters.

A comment on Apple M3 Ultra. Parent comment

Re:Apple M3 Ultra by bob1029 on Wednesday March 05, 2025 @02:06PM (#43270802)
> The question is if a llm will run with usable performance at that scale?

For the self-attention mechanism, memory bandwidth requirements scale ~quadratically with the sequence length.

[ Parent | on HN ]
Re:Apple M3 Ultra by kridsdale1 on Wednesday March 05, 2025 @02:36PM (#43271199)
Someone has got to be working on a better method than that. Hundreds of billions are at stake.
[ Parent | on HN ]