News for hackers. Stuff that matters.
Re:Web LLM – WebGPU Powered Inference of Large Language Models by KeplerBoy on Saturday April 15, 2023 @05:23PM (#35584670)
Doesn't it all boil down to bandwidth?

AMDs IGP are way less attractive because they use rather slow DDR4/5 memory while the M2 has blazing fast memory integrated in the package.

We're talking about 50 GB/s vs 400 GB/s. Nvidia's A100 has 1000 GB/s.

Memory bandwidth is usually the bottleneck in GPU performance as many kernels are memory-bound (look up the roofline performance model).

Re:Web LLM – WebGPU Powered Inference of Large Language Models by brucethemoose2 on Saturday April 15, 2023 @06:29PM (#35585255)
The AMD 6000 series has 128-bit LPDDR5 as an option, the 7000 series had LPDDR5X. This is similar to the M1/M2.

The Pro/Max have double/quadruple that bus width. But they are much bigger/more expensive chips.