News for hackers. Stuff that matters.

A comment on I think Anthropic and OpenAI have found product-market fit. Parent comment

Re:I think Anthropic and OpenAI have found product-market fit by girvo on Wednesday May 27, 2026 @06:06PM (#48301420)
DeepSeek v4 Flash, various quantised versions of Kimi K2.6, MiniMax 2.7, Qwen 3.5 “full sized, with a dual spark setup you can fit some decent setups on here

My single spark has me running Qwen 3.6 27B and antirez’s specially quantised DeepSeek v4 Flash (which is shockingly impressive)

[ Parent | on HN ]
Re:I think Anthropic and OpenAI have found product-market fit by Tepix on Wednesday May 27, 2026 @11:41PM (#48304214)
Kimi K2.6 does not run well on 256GB.
[ Parent | on HN ]
Re:I think Anthropic and OpenAI have found product-market fit by girvo on Thursday May 28, 2026 @03:23AM (#48305750)
True, I might be thinking of some of the communities four-Spark clusters for it; it’s already int4 right?
[ Parent | on HN ]
Re:I think Anthropic and OpenAI have found product-market fit by Tepix on Saturday May 30, 2026 @03:43AM (#48333721)
Yeah, the default quants are 595GB. Even four Sparks would require a quant lower than 4bit
[ Parent | on HN ]
Re:I think Anthropic and OpenAI have found product-market fit by zozbot234 on Thursday May 28, 2026 @05:23AM (#48306593)
Have you tried it? It would be slow for sure, but the main limitation AIUI would actually be storing the context in RAM - models like Kimi and GLM have high demands there which limit your ability to get meaningful aggregate throughput via large batches.
[ Parent | on HN ]
Re:I think Anthropic and OpenAI have found product-market fit by Tepix on Saturday May 30, 2026 @03:39AM (#48333696)
No need to try really. 1100b weights with 256GB RAM that‘s less than 1.8 bits per weight if you want a little bit of context.

How is that supposed to give good results?

[ Parent | on HN ]