Re:Beating GPT-5.6 Sol on retrieval with 100x cheaper open m...
by kumama on Wednesday August 05, 2026 @10:16PM
(#49191631)
castform founder here. the roi optimization makes sense. i think there are lots of usecases for which even a 2% gain in accuracy can be quite useful. off the top of my head
- high volume customer support. higher accuracy means fewer escalation, reducing labor costs - fraud detection. catching even one extra fraud attempt could mean a lot in savings - and ofc the classic ads use-case where at scale bps in improvement could mean millions in revenue :)
Re:Beating GPT-5.6 Sol on retrieval with 100x cheaper open m...
by simianwords on Thursday August 06, 2026 @03:30AM
(#49193651)
Would a bigger model be able to beat yours if some effort were put into prompt?
Re:Beating GPT-5.6 Sol on retrieval with 100x cheaper open m...
by dkersten on Thursday August 06, 2026 @06:03AM
(#49194686)
At what cost though? “Beating” isn’t enough if it costs 100x or even 10x the amount.
Re:Beating GPT-5.6 Sol on retrieval with 100x cheaper open m...
by kumama on Thursday August 06, 2026 @10:12AM
(#49196968)
the bigger model would still cost more :)
at the same time, i see prompting as being orthogonal to post-training. i'd imagine post-training a smaller model with a better prompt would make it perform even better
Re:Beating GPT-5.6 Sol on retrieval with 100x cheaper open m...
by simianwords on Thursday August 06, 2026 @02:29PM
(#49200483)
yes, can you show me tasks where this is true?
Smaller models are cheaper, sometimes faster. I agree that the “we’re an LLM fine-tuned for X” hasn’t worked out because you can just train Claude to do X (and Anthropic will), but not burning Opus/Fable tokens on dumb-but-token-heavy tasks is good sense.
As we move from “integrate AI into Y” to “optimize the ROI on Y”, we’ll see more of this.