I guess what you are saying would probably have been said by AI skeptics in the 70s, but LLMs provided a quantum leap. Yes, progress is often asymptotic and governed by diminishing returns, but discontinuous breakthrough must also be factored in.

Please tell me what quantum leap was provided by LLMs. Please inform me of any developments that made current LLMs possible.

I contend that there are none. Witness the actual transformer kernel technologies over the last 20 years and try to find a single new one.

Neural Networks? that's 90's technology. Scale is the only new factor I can think of.

This is an investor-dollar driven attempt to use brute-force to deliver "magical" results when the fundamental technology is being mis-represented to the general public, to CTOs, and even to Developers.

This is dishonest and will not end well.

Nailed it.

The biggest capability jump comes from semantic search. You can now search based on the content of a text rather than a literal character level match.

I am slowly coming 'round to the same conclusion: Word2Vec might be as fundamental as fire - all due caveats aside, of course ...

Technological advances tend to happen in jumps. We are no doubt approaching a local optima right now, but it won't be long until another major advancement propels things forward again. We've seen the same pattern in ML for decades.

Please name me one technological advance of major import in the fundamental transformer kernel space that has occurred in the last decade that has any import at all on today's LLMs.

I will wait.

The very idea of the Transformer architecture. Surely you've heard of "Attention is all you need".