Re:Any insider takes on Yann LeCun's push against current ar...
by haolez on Friday March 14, 2025 @06:39PM
(#43368066)
Not exactly related, but I wonder sometimes if the fact that the weights in current models are very expansive to change is a feature and not a "bug".
Somehow, it feels harder to trust a model that could evolve over time. It's performance might even degrade. That's a steep price to pay for having memory built in and a (possibly) self-evolving model.
Re:Any insider takes on Yann LeCun's push against current ar...
by bashfulpup on Friday March 14, 2025 @06:54PM
(#43368190)
We degrade, and I think we are far more valuable than one model.
Re:Any insider takes on Yann LeCun's push against current ar...
by nradov on Friday March 14, 2025 @06:13PM
(#43367830)
For a kid with a laptop to solve it would require the problem to be solvable with current standard hardware. There's no evidence for that. We might need a completely different hardware paradigm.
Re:Any insider takes on Yann LeCun's push against current ar...
by bashfulpup on Friday March 14, 2025 @06:53PM
(#43368179)
Also possible and a fair point. My point is that it's a "tiny" solution that we can scale.
I could revise that by saying a kid with a whiteboard.
It's an einstein×10 moment so who know when that'll happen.
These long horizon (agi) problems have been there since the very beginning. We have never had a solution to them. RL assumes we know the future which is a poor proxy. These energy based methods fundamentally do very little that an RNN didn't do long ago.
I worked on higher dimensionality methods which is a very different angle. My take is that it's about the way we scale dependencies between connections. The human brain makes and breaks a massive amount of nueron connections daily. Scaling the dimensionality would imply that a single connection could be scalled to encompass significantly more "thoughts" over time.
Additionally the true to solution to these problems are likely to be solved by a kid with a laptop as much as an top researcher. You find the solution to CL on a small AI model (mnist) you solve it at all scales.