I find these apparently hot takes like Booch and even Chomsky to be pretty shallow.

“We don’t really know how or why it works, but we are certain it doesn’t work like this”.

These systems are constantly being compared to perfectly rational humans, and getting the wrong answer seems to be taken as being a fault. But in a world of flat-earthers and truthers, why is the bar for general intelligence being set so high for LLMs? I struggled to follow the word puzzle and I’m not even a flat earther. And ISTM that gpt4’s treatment of a sippy cup revealed a problem with the question, not the answer.

Although I’m far from an expert, I find it difficult to understand why the brain couldn’t be just an optimised form of these LLMs. My (ignorant) supposition is that researchers will find shortcuts to these models that will have similarities the brain: for example, particular natural language structures (like subject-verb-object ordering) could be found to improve model efficiency in some unexpected way, reducing the cost of the models - something evolution does naturally. I mean, unless you believe the brain can’t be modelled mathematically, at some point we have to accept that we are approaching some kind of effective model.

It seems to me that the “it can’t be so” are being a bit unscientific. If there was hard evidence that LLMs were one way or another then it would be presented. Instead, currently all we seem to get is a bunch of opinions from Important People.

But they are appeals to authority, not science.

> These systems are constantly being compared to perfectly rational humans, and getting the wrong answer seems to be taken as being a fault. But in a world of flat-earthers and truthers, why is the bar for general intelligence being set so high for LLMs?

The input of this was a collection of data produced by humans. The by default behavior (if advanced) would presume to be human-like, and yet everyone seems to want to set the bar to be an all knowing oracle.

It's such a fundamental mistake people keep making.

There are a whole lot of humans out there. Not to mention there are 5 year-old humans, 3 year old humans. We seem unable to imagine for example a human child before developing theory of mind, but with an adult's vocabulary is still extremely human like. LLMs may not develop in conjunction with how humans develop, they may get some skills earlier some later. They may never get it from LLMs. But we need to get better at judging them for what they are capable, and not simply dismissing them because they aren't a perfect oracle nor a Phd grad.

Because you can't debate the AI. It's not on an open forum. People expect it to be an oracle because we interface with it as if it were.

> I find it difficult to understand why the brain couldn’t be just an optimised form of these LLMs

Axiomatically, Brains can't depended in their function on something they invented. People were hunting and skinning animals before learning to speak.

> in a world of flat-earthers and truthers, why is the bar for general intelligence being set so high

well, firstly, if you as a business are offering "here is an API to our AI, it has the intelligence of a drunk conspiracy nut" maybe that is not a viable business.

Second, there is surprising ability of human mind to go to work, do complicated tasks, drive a car, and still believe in dumb shit.

My point is that a lot of debate surrounding LLMs is based around the idea that the brain appears to have structures that are innate to language, but that LLMs don't (or at least, not to the same degree; ISTM that tokenisation is one such trick). And to me, this means that there may exist further mathematical optimisations that the brain is using that LLM's don't. Making it likely that we will some day be able to encode LLMs in less space/energy.

At which time I suppose we'll be told that LLMs can't be intelligent for some other reason.

Also, you can have high general intelligence and still be confidently wrong.

It definitely doesn’t have a comprehensive real world model like a human, but there’s probably something extra besides just statistics:

https://thegradient.pub/othello/

> They suggest language models can develop world models for very simple concepts in their internal representations (layer-wise activations), such as color [9], direction [10], or track boolean states during synthetic tasks [11]

But I don’t think there have been any probes like this for ChatGPT yet

This is better evidence than the Twitter example but I bet it still wouldn’t be well received in that thread

As a side note I just got GPT to write me some Go code and it came up with this:

    templates, err := embedFS.Sub("templates")

interestingly, this is not valid Go code (Sub is a function in the "fs" package, not a method) - but it would be quite typical in say Java.

So this mistake makes me think that GPT certainly does have a "mental model" for Go. And it's made the same kind of logical mistake I made when I first learned Go.

I wonder if it does better if you instruct it to 'pose' as a Go programmer. Or even one who comments to his younger Java-model self?

Except...no lol. Not perfect but Gpt-4's spatial reasoning is consistently solid and if you bothered to test it yourself, it'd be easy to see.

Also Appeal to authority is meh. When these guys can test for this special "true understanding" TM that gpt supposedly lacks then we'll talk. Otherwise it's just vague and ill defined assertions. A distinction you can't test for is not a distinction.

Depends on the modality of spatial reasoning. I tested it by asking it stroke-by-stroke instructions to write Chinese characters, and it gets something wrong every time and usually in an egregious way.

Grady only refute it but couldnt e plain or furnish proof. Is like FBI having the laptop and server data proof of wrong doing and then said no intent which isn't a thing in the context of broken law that should be charged. UML is one of the worst Software Engineering invented. I have in 6 different IT companies that basically throw away and internally banned UML. It is just too slow and treat all programming as solved like in the 80s and 90s. If you follow UML uou wont be able to create GPT in hundred years! Grady is outdated dino that shouldnt even been credited with anything.

He seems completely dismissive, and rather certain that GPT doesn't have a 'world model', despite IMHO not really having enough of an understanding (since no-one does, really), to justify that certainty.

GPT has been trained on text, and so any 'world model' is going to be a step removed from the world, compared to systems that have been trained on real-world stimulii (vision, sound, touch) like animals and humans.

But that does not mean it doesn't have a 'world model' at all, just that it's models are not going to be something we immediately recognize.