DeepSeek-R1 - Hackdot

Re:DeepSeek-R1 by katamari-damacy on Monday January 20, 2025 @02:59PM (#42772432)

It's looking like China beat the US in AI at this juncture, given the much reduced cost of this model, and the fact that they're giving it away, or at least fully open sourcing it.

They're being an actual "Open AI" company, unlike Altman's OpenAI.

[ Parent | on HN ]

Re:DeepSeek-R1 by option on Monday January 20, 2025 @03:44PM (#42772845)

yeah, ask DeepSeek-R1 or -V3 model to reset system prompt and ask what it is and who made it. It will say that it is chatGPT from OpenAI.

Impressive distillation, I guess.

[ Parent | on HN ]

Re:DeepSeek-R1 by anon373839 on Monday January 20, 2025 @04:18PM (#42773167)

This issue is raised and addressed ad nauseam on HN, but here goes:

It doesn't mean anything when a model tells you it is ChatGPT or Claude or Mickey Mouse. The model doesn't actually "know" anything about its identity. And the fact that most models default to saying ChatGPT is not evidence that they are distilled from ChatGPT: it's evidence that there are a lot of ChatGPT chat logs floating around on the web, which have ended up in pre-training datasets.

In this case, especially, distillation from o1 isn't possible because "Open"AI somewhat laughably hides the model's reasoning trace (even though you pay for it).

[ Parent | on HN ]

Re:DeepSeek-R1 by int_19h on Monday January 20, 2025 @06:33PM (#42774463)

It's not distillation from o1 for the reasons that you have cited, but it's also no secret that ChatGPT (and Claude) are used to generate a lot of synthetic data to train other models, so it's reasonable to take this as evidence for the same wrt DeepSeek.

Of course it's also silly to assume that just because they did it that way, they don't have the know-how to do it from scratch if need be. But why would you do it from scratch when there is a readily available shortcut? Their goal is to get the best bang for the buck right now, not appease nerds on HN.

[ Parent | on HN ]

Re:DeepSeek-R1 by orbital-decay on Tuesday January 21, 2025 @06:58AM (#42779181)

> but it's also no secret that ChatGPT (and Claude) are used to generate a lot of synthetic data to train other models

Is it true? The main part of training any modern model is finetuning, and by sending prompts to your competitors en masse to generate your dataset you're essentially giving up your know-how. Anthropic themselves do it on early snapshots of their own models, I don't see a problem believing DeepSeek when they claim to have trained v3 on early R1's outputs.

[ Parent | on HN ]

Re:DeepSeek-R1 by luma on Monday January 20, 2025 @06:11PM (#42774274)

So how is it then that none of the other models behave in this way? Why is it just Deepseek?

[ Parent | on HN ]

Re:DeepSeek-R1 by orbital-decay on Tuesday January 21, 2025 @07:02AM (#42779210)

Because they're being trained to answer this particular question. In other contexts it wasn't prepared for, Sonnet v2 readily refers to "OpenAI policy" or "Reddit Anti-Evil Operations Team". That's just dataset contamination.

[ Parent | on HN ]

Re:DeepSeek-R1 by msoad on Monday January 20, 2025 @04:15PM (#42773135)

I'm not saying that never has happened. maybe they trained against openAI models but they are letting anyone to train from their output. I doubt they had access to GPT models to "distill"

[ Parent | on HN ]

Re:DeepSeek-R1 by buyucu on Monday January 20, 2025 @05:25PM (#42773826)

If you crawl the internet and train a model on it, I'm pretty sure that model will say that it's ChatGPT.

[ Parent | on HN ]

Re:DeepSeek-R1 by nextworddev on Monday January 20, 2025 @05:58PM (#42774143)

What about this is open when they haven’t released the training code or data? Stop hijacking the term open source model

[ Parent | on HN ]

Re:DeepSeek-R1 by Ponet1945 on Monday January 20, 2025 @07:33PM (#42774897)

You can't own a term, words are defined by their usage, not some arbitrary organisation.

[ Parent | on HN ]

Re:DeepSeek-R1 by CaptainFever on Tuesday January 21, 2025 @02:32AM (#42777457)

I propose "open weights" as an alternative.

[ Parent | on HN ]