Re:Next-Concept Modeling: Fast, Parallel Coarse-to-Fine Text...
by LoganDark on Monday November 09, 2026 @12:34PM
(#50428702)
> The problem here is that they aren't equivalent platforms - no amount of money will buy you an RTX 5090 with 512 GB of RAM.
I agree that Apple silicon is far better in terms of memory capacity, which is part of why I'm rooting for it. But an SoC like Apple silicon is roughly analogous to a CPU+GPU when it comes to compute, and the GPU side is quite lacking compared to today's "traditional" GPUs, even though the CPU is somewhat leading the industry.
Mainly, I'm disappointed with the performance of dense LLMs on Apple silicon compared to Nvidia, due to the large disparity in both memory bandwidth and GPU compute. Apple's latest chips are starting to push into just over a terabyte per second of bandwidth -- about two-thirds of what the 5090 achieved nearly two years ago now. They are catching up quick and I do hope they pass, but they haven't quite yet.
I agree that Apple silicon is far better in terms of memory capacity, which is part of why I'm rooting for it. But an SoC like Apple silicon is roughly analogous to a CPU+GPU when it comes to compute, and the GPU side is quite lacking compared to today's "traditional" GPUs, even though the CPU is somewhat leading the industry.
Mainly, I'm disappointed with the performance of dense LLMs on Apple silicon compared to Nvidia, due to the large disparity in both memory bandwidth and GPU compute. Apple's latest chips are starting to push into just over a terabyte per second of bandwidth -- about two-thirds of what the 5090 achieved nearly two years ago now. They are catching up quick and I do hope they pass, but they haven't quite yet.