News for hackers. Stuff that matters.
Re:Yann LeCun: ChatGPT is 'not particularly innovative' by belval on Thursday January 26, 2023 @02:38PM (#34537554)
> OpenAI's really interesting approach to GPT

That's the issue though, Yann LeCun is specifically referring to ChatGPT as the standalone model, not the GPT family since a lot of models at Meta, Google, DeepMind are based on a similar approach. His point is that ChatGPT is a cosmetic additional training with prompt with a nice interface, but not a fundamentally different model than stuff we've have had for +2-3 years at this point.

Re:Yann LeCun: ChatGPT is 'not particularly innovative' by cl42 on Thursday January 26, 2023 @03:07PM (#34537988)
ChatGPT is build on GPT-3. GPT-3 was a big NLP development. The paper has 7000+ citations: https://arxiv.org/abs/2005.14165 It was a big deal in the NLP space.

It wasn't a 'cosmetic' improvement over existing NLP approaches.

Re:Yann LeCun: ChatGPT is 'not particularly innovative' by belval on Thursday January 26, 2023 @03:15PM (#34538101)
Respectfully I don't think you read my comment. GPT3 != ChatGPT. ChatGPT is built on GPT-3 and is not breaking new ground. GPT3 is 3 years old and was breaking new ground in 2020 but Meta/Google/DeepMind all have LLM of their own which could be turned into a Chat-Something.

That's the point LeCunn is making. He's not out there negating that the paper you linked was ground-breaking, he's saying that converting that model into ChatGPT was not ground-breaking from an academic standpoint.

Re:Yann LeCun: ChatGPT is 'not particularly innovative' by cl42 on Thursday January 26, 2023 @03:20PM (#34538180)
@belval -- sorry, can't reply directly. I understand what you're saying -- fair enough! I appreciate the clarification.