There is some ability for it to make novel connections but it's pretty small. You can see this yourself having it build novel systems.

It largely cannot imaginr anything beyond the usual but there is a small part that it can. This is similar to in context learning, it's weak but it is there.

It would be incredible if meta learning/continual learning found a way to train exactly for novel learning path. But that's literally AGI so maybe 20yrs from now? Or never..

You can see this on CL benchmarks. There is SOME signal but it's crazy low. When I was traing CL models i found that signal was in the single % points. Some could easily argue it was zero but I really do believe there is a very small amount in there.

This is also why any novel work or findings is done via MASSIVE compute budgets. They find RL enviroments that can extract that small amount out. Is it random chance? Maybe, hard to say.

Is this so different from what we see in humans? Most people do not think very creatively. They apply what they know in situations they are familiar with. In unfamiliar situations they don't know what to do and often fail to come up with novel solutions. Or maybe in areas where they are very experienced they will come up with something incrementally better than before. But occasionally a very exceptional person makes a profound connection or leap to a new understanding.

Sure we make small steps at the time but we compound these unlike AI.

AI cannot compound their learnings for the foreseeable future

This is definitely wrong, most AI researchers DO NOT agree with LeCun.

Most ML researchers think AGI is imminent.

Where do you get your majority from?

I don't think there is any level of broad agreement right now. There are tons of random camps none of which I would consider to be broadly dominating.

Who is in this group of ML researchers?

People with OpenAI shares, probably.

The ones being paid a million dollars a year by OpenAI to say stuff like that, maybe.

The guy who built chatgpt literally said we're 20 years away?

Not sure how to interpret that as almost imminent.

> The guy who built chatgpt literally said we're 20 years away?

20 years away in 2026, still 20 years away in 2027, etc etc.

Whatever Altman's hyping, that's the translation.

Do you have poll of ML researchers that shows this?

Well, can you point us to their research then? Please.

their employment and business opportunities depend on the hype, so they will continue to 'think' that (on xitter) despite the current SOTA of transformers-based models being <100% smarter than >3 year old GPT4, and no revolutionary new architecture in sight.

You're going to be in for a very rude awakening.

Do you have a pointer to where LeCun spoke about it? I noticed last October that Dwarkesh mentioned the idea off handedly on his podcast (prompting me to write up https://manifold.markets/MikeLinksvayer/llm-trained-on-data-...) but I wonder if this idea has been around for much longer, or is just so obvious that lots of people are independently coming up with it (parent to this comment being yet another)?

Preface: Most of my understand of how LLMs actually work comes from 3blue1brown's videos, so I could easily be wrong here.

I mostly agree with you, especially about distrusting the self-interested hype beasts.

While I don't think the models are actually "intelligent", I also wonder if there are insights to be gained by looking at how concepts get encoded by the models. It's not really that the models will add something "new", but more that there might be connections between things that we haven't noticed, especially because academic disciplines are so insular these days.

How about this for an evaluation: Have this (trained-on-older-corpus) LLM propose experiments. We "play the role of nature" and inform it of the results of the experiments. It can then try to deduce the natural laws.

If we did this (to a good enough level of detail), would it be able to derive relativity? How large of an AI model would it have to be to successfully derive relativity (if it only had access to everything published up to 1904)?

I don't know if any dataset of pre 1904 writing would be large enough to train a model that would be smart enough. I suspect that current sized SOTA models would at least get to special relativity, but for general relativity and quantum mechanics I am less sure.

What do they (or you) have to say about the Lee Sedol AlphaGo move 78. It seems like that was "new knowledge." Are games just iterable and the real world idea space not? I am playing with these ideas a little.

AlphaGo is not an LLM

And? Do the arguments differ for LLM vs the other models?

I guess the arguments sometimes mention languages. But I feel like the core of the arguments are pretty much the same regardless?

The discussion is about training an LLM on old text and then asking it about new concepts.

AlphaGo was trained on old games and then presented with a game it had never seen before. It came up with a move that a human would not have played.