>> We definitely haven't hit the ceiling and everyone's noticing "Hey this Opus 5.5 with the increased parameters seems much better than the last one".
Opus is better (not necessarily "very much") than Sonnet in coding. That's because of a gigantic engineering effort to make it better at coding. That is a regression: we've gone from "sparks of AGI" and systems that were said to be "genearl-purpose" to systems that do one thing well and everything else... well, there's no real measure of how better Opus is from Sonnet in replying to your emails so it's hard to say.
Same goes for OpenAI in maths. Industry wide, there is a regression from general-purpose to special-purpose systems, like in Good, Old-Fashioned AI. LeCun is absolutely right to complain that this is not the path to general intelligence. I don't think his preferred approach, JEPA, is going anywhere either, but at least he's trying. Other luminaries of the field have expressed similar doubts, e.g. Richard Sutton's tweet was in my news feed yesterday saying that brute force is ngmi and algorithmic advances are needed [1]. Schmidhuber I think has said multiple times something along the lines of "there is no intelligence without physical intelligence".
We are very far from that. The recent trend for Vision-Language-Action Transformers (VLA-Transformers) in robotics has conclusively demonstrated that you can't solve the real world by overfitting to a set of discrete tasks, and there is no source of data that we can harness to repeat the feat of LLMs. Algorithmic advances will be necessary, but as the article suggests, these have now be put on hold.
There is a dumbing down, slowing down of research as all resources are spent to try and scale what we have now. What we have now is not enough for intelligence, but that doesn't matter. The industry's goal is not intelligence, it's making more money than anyone has ever made before; in fact more money than all humans put together have ever made before. That's the current race: who will win the money game. Obviously that's not likely to do anything for intelligence.
____________
[1] Brute force computation is not enough for intelligence. Clever new methods are needed. The science of AI is far from finished. IMHO
Opus is better (not necessarily "very much") than Sonnet in coding. That's because of a gigantic engineering effort to make it better at coding. That is a regression: we've gone from "sparks of AGI" and systems that were said to be "genearl-purpose" to systems that do one thing well and everything else... well, there's no real measure of how better Opus is from Sonnet in replying to your emails so it's hard to say.
Same goes for OpenAI in maths. Industry wide, there is a regression from general-purpose to special-purpose systems, like in Good, Old-Fashioned AI. LeCun is absolutely right to complain that this is not the path to general intelligence. I don't think his preferred approach, JEPA, is going anywhere either, but at least he's trying. Other luminaries of the field have expressed similar doubts, e.g. Richard Sutton's tweet was in my news feed yesterday saying that brute force is ngmi and algorithmic advances are needed [1]. Schmidhuber I think has said multiple times something along the lines of "there is no intelligence without physical intelligence".
We are very far from that. The recent trend for Vision-Language-Action Transformers (VLA-Transformers) in robotics has conclusively demonstrated that you can't solve the real world by overfitting to a set of discrete tasks, and there is no source of data that we can harness to repeat the feat of LLMs. Algorithmic advances will be necessary, but as the article suggests, these have now be put on hold.
There is a dumbing down, slowing down of research as all resources are spent to try and scale what we have now. What we have now is not enough for intelligence, but that doesn't matter. The industry's goal is not intelligence, it's making more money than anyone has ever made before; in fact more money than all humans put together have ever made before. That's the current race: who will win the money game. Obviously that's not likely to do anything for intelligence.
____________
[1] Brute force computation is not enough for intelligence. Clever new methods are needed. The science of AI is far from finished. IMHO
https://x.com/RichardSSutton/status/2119142310894334262?a=ry...