Re:Anthropic S-1
by matteoraso on Monday October 26, 2026 @01:51PM
(#50247951)
I imagine that there's a lot of variance there based on what frontier model you want to run. Besides that, there's probably economies of scale that make it difficult to really ask how much it costs to run a single instance.
Re:Anthropic S-1
by kerwioru9238492 on Monday October 26, 2026 @01:39PM
(#50247749)
What do you consider a frontier model + what speeds do you want them to run at? These two questions need to be answered to answer your question.
A 3T frontier model can be run on 4 maxxed out Mac studios. But only for a single user.
> What is the hardware cost to run a single instance of a frontier model?
The latest Kimi model, which scores higher than 5.1 Fable on Artificial Analysis, is estimated to require around 6-7 TB of VRAM to run. If we are going with many A100 GPUs for this, then we are looking around a million dollars in GPUs alone
Re:Anthropic S-1
by colingauvin on Monday October 26, 2026 @09:26PM
(#50253613)
What? We know exactly what it takes to run: 2.8 TB VRAM for FP8...1.4 TB VRAM for FP4.
I think it comes down to: Will the hardware cost for frontier models come down?
Also, here's a business idea: Web crawler for open weight models. You just pay a monthly fee so that your home model can continuously update on a large web crawl each hour/day.