DeepSeek and the Effects of GPU Export Controls

Re:DeepSeek and the Effects of GPU Export Controls by sanjams on Thursday January 23, 2025 @10:26AM (#42804926)

I have read the R1 paper. My observation is that there is no information whatsoever about how they are overcoming the limitations of the H800 compared to the H100 which is what the parent article is about. That's the piece Im curious about.

I will concede that I have not read all their papers or looked through their code, but that's why I asked the question: I hoped someone here might be able to point me to specific places in specific papers instead of a axvix search.

[ Parent | on HN ]

Re:DeepSeek and the Effects of GPU Export Controls by lhl on Thursday January 23, 2025 @11:55AM (#42805655)

Give Section 3 of the DeepSeek-V3 paper a read. The discuss their HAI-LLM framework and have a pretty in-depth description of their DualPipe algorithm and how it compares to other pipeline bubbles. They also describe how they work around NVLink limits and tons of other optimizations in extreme depth. The section is 10 pages long, and it's relatively dense, not fluff!

[ Parent | on HN ]