> The current solution of 500b parameters or whatever is literally just a lookup table of existing data. They could replace it with an SQLite database with the same amount of data and get the same solution.

I'm pretty sure this is completely false?

Correct.

ML papers always try to find the most similar examples in the training set, and rarely find a close match.

How so? What would be the difference? Other than being unviable.

I don’t have a deep understanding of how these language models work, but what do I know makes think that saying “It is basically a lookup table” is sort of like saying “A car is basically a bike”.

Sure, there are some very vague similarities, but they are also incredibly different.

A bike and a car both take you places. But the kinds of places they can take you and how effectively they do so differs vastly.

Likewise, both lookup tables and LLMs take an input and then spit something out. But the similarities basically stop there.

I don't know about that - I've spend 100+ hours asking it to write Plays, TV Scripts, Poems, and lots of code (with Unit tests) - and it's pretty remarkable. Makes far fewer errors in simple stuff than I would - and way faster - I just need to proof it and copy-paste a lot of the time.

What I find amazing is when I ask it to do something like "Write a TV Script about Malcom Saving Kaylee from Aliens" - and then it fills in all the details of the Serenity, places Kaylee in the engine bay , has Mal firing off his pistols, and then retreating back to the Bridge".

Then, I simply type, "Add Elves" - and it rewrites the scenario, with the Elves properly using bows to fight off the aliens, and (this blows my mind), when they retreat to the bridge and Kaylee is worried about the engine, the Elves coherently say that it's okay, that they will use Magic to fix the engine and make it faster - And everything is written far more coherently than I ever would and makes absolute sense.

I get that it's a LLM - but damn, it's been blowing my mind for over a month, and I'm still astonished by it every day. I'm starting to wonder if my own intellect is anything more than an LLM connected to emotions/sense-organs.

That does not mean it's not terribly useful. The computer that helped man land on the moon couldn't reason either but it had a good enough model of the physics and a brand new way to fuse a bunch of sensors into a model of where the craft was. I think of these products as being tools guided by the smart human.