DeepSeek open-sources inference optimizations with 60–85% faster generation [pdf]

Re:DeepSeek open-sources inference optimizations with 60–85%... by Jackobrien on Saturday June 27, 2026 @06:02AM (#48696810)

I see a world soon where there’s an extremely wide variety of small models for speculative decoding, unique to use cases, companies, and even individuals.

[ Parent | on HN ]

Re:DeepSeek open-sources inference optimizations with 60–85%... by nicce on Saturday June 27, 2026 @06:13AM (#48696891)

Hopefully that is the case and hardware does not get impossible to get.

[ Parent | on HN ]

Re:DeepSeek open-sources inference optimizations with 60–85%... by pydry on Saturday June 27, 2026 @06:24AM (#48696953)

yes, heavily constrained by sophisticated guardrails.

this is definitely where things are going. the enormous "eat the world" models have extreme diminishing returns by comparison.

[ Parent | on HN ]

Re:DeepSeek open-sources inference optimizations with 60–85%... by Der_Einzige on Saturday June 27, 2026 @10:54AM (#48698828)

You clearly didn't read the recent speculative decoding papers because it's been possible to use any model to speculate for any other model for awhile. They solved the tokenization problems that prevented this in the past.

[ Parent | on HN ]