News for hackers. Stuff that matters.
Re:Accelerating GPT-5.6 Sol Ultrafast by YetAnotherNick on Thursday August 13, 2026 @04:56PM (#49291742)
LLama 3 405B had the most unoptimized kv cache usage by far. Deepseek v4 pro uses 2.4GB for the same context length[1].

[1]: https://vllm.ai/blog/2026-04-24-deepseek-v4

Re:Accelerating GPT-5.6 Sol Ultrafast by philipportner on Friday August 14, 2026 @04:57AM (#49296217)
Good point, thanks! I haven't been keeping up with most of the new model internals.