You're look at about 100 tokens/s for a 1T MoE 37B active 4bit model.
It'd probably cost $30k or more I'm guessing if memory prices do not come down. Even at $30k, it could still be a relative bargain since an RTX Pro 6000 Blackwell 96GB card costs $12k today. The M3 Ultra with 512GB was around $8k before Apple discontinued it. I expect an M7 Ultra to have 768GB or 1024GB.
Apple Silicon Macs were on their way to becoming cheap local LLM machines relative to professional GPUs before this memory crisis. It may still emerge as such in a few years.
Here's some interesting math: At 512GB, an Ultra chip could make 42 pro iPhones. Assume a 55% profit margins, and $1200 ASP, you're looking at $28,160 in profit from making iPhones instead. No wonder Apple discontinued the M3 Ultra 512GB. If they only have a limited supply of RAM for all their products, it makes no sense to produce an $8000 M3 Ultra 512GB when you can produce 42 pro iPhones. You can only configure an M3 Ultra up to 96GB today as of June 2026.
Apple would have to raise the price of a 512GB Ultra Mac to around $50k to match iPhone profits.
How would that work? They purchase 512GB from Samsung and then it doesn't matter if that's like 128x 4GB or 4x 128GB?
If comapnies keep spending half a macbook neo worth of subscription on AI plans monthly per person, Apple is going to have a hard time competing.
Edit: for those of you downvoting I don’t celebrate this prospect. I’m merely realistic about where things are going given the rapid vibe shift from the administration on AI since the start of June.
That’s indeed very hypothetical considering that Apple silicon uses on-package HBM.
The base model was $9k, that much RAM got you into $14k range.
(This is assuming Apple will deliver, but this area is one of the biggest ones they have in AI, and they need the developer ecosystem to exist and survive)
M1 had 70 GB/s, M1 Pro: 200, M1 Max 400, M1 Ultra 800.
Modern RTX 6000: ~1,600 or so.
If we get a 1,200-1,500 GB/s bandwidth M7 variant in late 2027 with 512GB of RAM, that will be a very interesting chip. Tracking LLM size and performance improvements, I can imagine that being a sort of inflection point for local inference. I wonder what the power budget would be in desktop format.