News for hackers. Stuff that matters.

A comment on The AMD Radeon Instinct MI300A's Giant Memory Subsystem. Parent comment

Re:The AMD Radeon Instinct MI300A's Giant Memory Subsystem by sroussey on Saturday January 18, 2025 @12:48PM (#42749979)
A Mac with that memory will have closer to 500GB/s but your point still stands.

That said, if you just want to play around, having more memory will let you do more interesting things. I’d rather have that option over speed since I won’t be doing production inference serving on my laptop.

[ Parent | on HN ]
Re:The AMD Radeon Instinct MI300A's Giant Memory Subsystem by lhl on Saturday January 18, 2025 @12:54PM (#42750021)
Yeah, the M4 Max actually has pretty decent MBW - 546 GB/s (cheapest config is $4.7K on a 14" MBP atm, but maybe there will be a Mac Studio at some point). The big weakness for the Mac is actually the lack of TFLOPS on the GPU - the beefiest maxes out at ~34 FP16 TFLOPS. It makes a lot of use cases super painful, since prefill/prompt processing can take minutes before token generation starts.
[ Parent | on HN ]