Re:The AMD Radeon Instinct MI300A's Giant Memory Subsystem
by enragedcacti on Saturday January 18, 2025 @10:53AM
(#42749201)
> Of course, if it's not real VRAM, it needs to be at least somewhat close on the latency and bandwidth front
It is close to VRAM*, just not close to DRAM on a conventionally designed CPU. This thing is effectively just a GPU that fits in a CPU slot and has CPU cores bolted to the side. This approach has the downside of worse CPU performance and the upsides of orders of magnitude faster CPU<->GPU communication, simpler programming since coherency is handled for you, and access to substantial amounts of high bandwidth memory (up to 512GB with 4 MI300As).
It is close to VRAM*, just not close to DRAM on a conventionally designed CPU. This thing is effectively just a GPU that fits in a CPU slot and has CPU cores bolted to the side. This approach has the downside of worse CPU performance and the upsides of orders of magnitude faster CPU<->GPU communication, simpler programming since coherency is handled for you, and access to substantial amounts of high bandwidth memory (up to 512GB with 4 MI300As).
* https://chipsandcheese.com/p/microbenchmarking-nvidias-rtx-4...