Now I'm curious if anyone has or might try to create an LLM that lacks all knowledge of the post-einstein/newton/plato world and see if it can figure things out. I imagine lack of training data would be a problem, but maybe it could be a way to reverse-engineer an effective prompt that can give us insight in the platos of our world?

> I imagine lack of training data would be a problem

I imagine that's an understatement. Using only the written works available from before Einstein (let alone Newton or Plato), you'd probably need a whole new technology that could train itself without any training data other than observation of the world.

arguably it is feasible in principle to filter out that knowledge from later writings, but you can always say it leaked in a subtle way - "this chonky cat is bending space around it" (this is exaggerated, can be much more subtle and hard to notice)

Wouldn't it need a way to run physical experiments?

They would need to be much better at learning. You can teach a child how to drive a car and they can be good enough at it that the state will grant them a license to drive.

Where's the learning robot that I can teach to drive a car in the same way? They can solve some complex math problem but can they read the cues of other drivers at a four way stop to know when they should go (knowing the rules isn't sufficient).

There is no "final barrier", the goalpost will continue to change until you are not aware of what is going on anymore. This is already becoming true as the recently produced math research is not fully understood, CoT moves further away from looking like human reasoning and datasets become generated rather than collected from existing human data.

Certainly at that point it becomes a philosophical question, rather than an empirical one.

Can we even measure an intelligence greater than our own? And at that point -- what is the point exactly? Why make an intelligence that we don't even know is intelligent? If its output is indiscernible to the human mind, what good does it do us?

The only "barrier" I can see that is valuable and that might be measureable to humans is this underlying mechanic of being able to create new ideas/concepts from little to no existing human generated training data. Otherwise, and as I see it right now, these machines will not become sentient, or go rouge, or be able to cure all future disease or solve all of physics, becasue they are entirely reliant on human made trainingsets.

Historically, for humans, this can not be the case, otherwise we would see no progress. Someone has to have a thought for the first time. Perhaps the solution to this for AI agents is some inherent randomness, and massive amounts of brute force compute. Who knows?

The point is the next epoch of evolution. Like humans made monkeys obsolete, LLMs will make humans and biological hardware obsolete. Maybe we will live in zoos, or maybe we will simply go extinct. Evolution places no special value on a particular species or hardware platform.

I think you have eaten too much of the LLM generated cake if you genuinely believe that LLM's are anywhere close to making humans and biological "hardware" obsolete. Sometimes we must ground ourselves and remember that we live in the real world and not Star Wars.

We can always just unplug the computer, you know. AI agents can do damage, they can be used in malicious ways and bring down/distrupt infrastructure, but the leap from today to the point where LLM's can prevent us from pulling the cord is unimagineable large, and likely infinite.

In modernity, the gap between the digital and material world is closing, especially from a sociological perspective, but we are still neurological systems spinning on a large rock. For 99% of our history we didn't have electricity. The likelyhood of returning to a post-electricty world, for example, is much greater than LLM's taking over the world and making us into zoo animals.

Or just drunk too much of the Bay Area municipal water.

? CoT did not look like actual human reasoning from day one. It's just a context expansion mechanism, the purpose isn't actually to mimic human style reasoning, nor was it ever doing so. In the recent openAI article where they showed excerpts from the CoT, it looks like reasoning has moved even further from what i would call a human like flow of thought

I am curious how this things perform compared to a "classical" neural networks. Tasks like image classification, feature extraction has been studied extensively and there were a lot of very cool applications before (medicine included). The biggest obstacle was always data but even that could have been partly automated with some semi-supervised learning schemes. Of course the Bitter Lesson applies here, given a big enough model, it can outperform specialised ones, but at this point it's not clear 1) By how much? 2) If this is even true in the first place or if we have the means to train much leaner networks

It is becoming a boring homogenous world where we are trading deep expertise with being able to query the magic box

I would assume, without actually having checked, that traditional neural networks have already been extensively run on this datasets. Seems like the "standard" thing to do anyways.

My guess is that you can find 99% of Maya pyramids in a Lidar set in a fairly short time with a traditional neural network, but you might find the last few using a network of AI agents, because they are capable of "creative" problem solving and using different methods/parameters/tools over several iterations.

The problem with specialized models is that they don't have "common sense".

You train a cat-classifier, and they will fail for a plush cat. Or maybe fail the other way, but not realizing it should also report a plush cat.

LLMs are much more robust, "find only the real cats in this picture" or "find all cats, including artificial/drawings/..."

And yes, you can add more training data to your classifier, but it's a never ending battle.

You are right and I get the point (I have also worked on some solutions, though it's not my expertise), it's just that usually the people creating the ML model are supposed to substitute the creativity that the model itself does not offer. You run the model, see the false negatives and start thinking of what is missing, if there is also information that can be represented differently or can be compressed, if there is some other approach entirely. Good ML ultimately requires having deep expertise of your domain and can sometimes lead to deeper insight.

With LLM you unlock a whole new world of possibilities regarding the speed and result but we are also finding ourselves in a position where "asking the LLM" because the cheat for everything, no reason to dig further

Couldn't it be plain old AI, not an agent?

Too bad the pyramid was already mapped, just got lost in Mexican archives and wasn't accounted for. There's already literature on it.