Re:Nvidia, ASML Plunge as DeepSeek Triggers Tech Stock Selloff
by thot_experiment on Tuesday January 28, 2025 @02:57AM
(#42849932)
OP probably means "the largest distilled model"
[ Parent
| on HN ]
Re:Nvidia, ASML Plunge as DeepSeek Triggers Tech Stock Selloff
by menaerus on Tuesday January 28, 2025 @03:36AM
(#42850146)
So not an actual DeepSeek-R1 model but a distilled Qwen or Llama model.
From DeepSeek-R1 paper:
> As shown in Table 5, simply distilling DeepSeek-R1’s outputs enables the efficient DeepSeekR1-7B (i.e., DeepSeek-R1-Distill-Qwen-7B, abbreviated similarly below) to outperform nonreasoning models like GPT-4o-0513 across the board.
and
> DeepSeek-R1-14B surpasses QwQ-32BPreview on all evaluation metrics, while DeepSeek-R1-32B and DeepSeek-R1-70B significantly exceed o1-mini on most benchmarks.
and
> These [Distilled Model Evaluation] results demonstrate the strong potential of distillation. Additionally, we found that applying RL to these distilled models yields significant further gains. We believe this warrants further exploration and therefore present only the results of the simple SFT-distilled models here.