Closed AI models dominate the price-performance frontier

Re:Closed AI models dominate the price-performance frontier by ozgrakkurt on Friday November 06, 2026 @05:14AM (#50390472)

How do you even fit that model in there? What quant is it? Are you using llama.cpp?

[ Parent | on HN ]

Re:Closed AI models dominate the price-performance frontier by notenchange on Friday November 06, 2026 @05:53AM (#50390768)

No, I'm using strata. This is a MoE model, so it keeps the last used model in VRAM, and the rest in RAM. If it needs, it swaps the less used model in VRAM for the model in RAM for the next token.

Seems very slow, but it's actually fine.

[ Parent | on HN ]

Re:Closed AI models dominate the price-performance frontier by notenchange on Friday November 06, 2026 @09:44AM (#50392927)

P.S. Sorry, where you read “models” I meant experts.

[ Parent | on HN ]

Re:Closed AI models dominate the price-performance frontier by ozgrakkurt on Friday November 06, 2026 @11:46PM (#50401702)

Thanks, the information in their docs are really promising. I have been meaning to try it but wasn't able to yet.

[ Parent | on HN ]