somnial
What happens when a GPU reads memory?
The case for disaggregated LLM serving
On-the-fly snapshot compression for elastic inference at scale
Re:DeepSeek V4 Flash on a Single AMD MI300X
on Tuesday August 04, 2026 @12:43PM (#49171387)
throughput scales superlinearly with number of GPUs when networked well and deployed with wideEP, so 1x won't compare.
also it would be interesting to figure from the DSpark paper whether their numbers are consistent with the GPUs still being H800s, since they never actually say...
NVLink, NVSwitch, and All That
The Anatomy of an Instruction Pipeline Hazard
Width vs. Depth: Speculating on the Margin
Re:The gap between open weights LLMs and closed source LLMs
on Saturday June 27, 2026 @12:55PM (#48699802)
this is a blog post from a company that hosts open weights LLMs (https://www.doubleword.ai/). I think its possible it might have been tongue in cheek
Re:The Economics of Speculative Decoding
on Thursday June 11, 2026 @11:27AM (#48491672)
https://fergusfinn.com/blog/economics-of-speculative-decodin...
good point tho - plus for Deepseek the shared expert increases the overlap slightly
Re:Speculative KV coding: losslessly compressing KV cache by up to ~4×
on Monday June 08, 2026 @05:51AM (#48443249)
true, but no reason the predictor model couldn't use linear attention (i.e. mamba, GDN etc) to predict KV caches