News for hackers. Stuff that matters.

A comment on DeepSeek just open-sourced its inference engine. Parent comment

Re:DeepSeek just open-sourced its inference engine by freakynit on Tuesday October 20, 2026 @06:25AM (#50163958)
It's exceptionally optimized... 890 bytes per token. At standard 32K context, that's less than 30MB.

1. https://huggingface.co/deepseek-ai/DeepSeek-V4.1-Flash

2. https://www.mindstudio.ai/blog/deepseek-v4-1-flash-specs-arc...

[ Parent | on HN ]
Re:DeepSeek just open-sourced its inference engine by entrope on Tuesday October 20, 2026 @06:57AM (#50164340)
Sure, but that size/token is inherent to the model. Part of DeepSeek's hosting advantage, and thus a remaining part of their moat, is migrating KV cache from VRAM to host RAM to nonvolatile storage, and presumably routing requests to where KV cache has migrated. People here on HN say KV cache lifetimes can be 8+ hours with DeepSeek, and I don't know of any other provider that can match that. That infrastructure isn't part of this open-source release.
[ Parent | on HN ]
Re:DeepSeek just open-sourced its inference engine by freakynit on Tuesday October 20, 2026 @07:06AM (#50164419)
That's right.
[ Parent | on HN ]
Re:DeepSeek just open-sourced its inference engine by _aavaa_ on Tuesday October 20, 2026 @06:57AM (#50164335)
Yes, the theoretical approach is listed there. But the actual code is what's missing.
[ Parent | on HN ]