News for hackers. Stuff that matters.

A comment on Apple M3 Ultra. Parent comment

Re:Apple M3 Ultra by musicale on Thursday March 06, 2025 @12:49AM (#43276826)
> so you had to have lots of operand re-use to not be memory-bound

Looking at Nvidia's spec sheet, an H100 SXM can do 989 tf32 teraflops (or 67 non-tensor core fp32 teraflops?) and 3.35 TB/s memory (HBM) bandwidth, so ... similar problem?

[ Parent | on HN ]
Re:Apple M3 Ultra by pklausler on Thursday March 06, 2025 @01:29AM (#43277059)
There is caching today.
[ Parent | on HN ]
Re:Apple M3 Ultra by ryao on Thursday March 06, 2025 @09:52AM (#43280896)
The cache hitrate is effectively 0 for LLMs since the datasets are so huge.
[ Parent | on HN ]