Maybe it is human like intelligence already. Maybe our internal monologue is just a better trained and refined ChatGPT. And maybe that is all the magic that is necessary for this holy grail of consciousness, there is no quantum brain, no nothing. Just a stream of the next word that says that we are there, therefore we are. That is what scares me.

> Maybe it is human like intelligence already.

It objectively is not.

First, intelligence (human or otherwise) is typically defined as the sum of value functions of a given policy across all states, divided by an exponential of the complexity of the environment[0] (page 23). The GPT-4 technical report[1] specifically points out that it is unable to make money when given an initial amount and AWS access (appendix H § 2.9). A human can.

Second, let's look at it mechanically. It is very easy to anthropomorphize machines, but they are just built differently. Neurons fundamentally are not approximated by self-attention and it is highly unlikely that they improve through backpropagation. In terms of behaviour, we can and do notice differences. Transformers can, and do, make logical mistakes; when asked to immediately answer a complex question, they can only think through it for a fixed amount of time, and thus fail to give an accurate answer. More importantly, they just don't care about their (or your) own survival; they can give lethal advice. All it takes is to find a prompt for which the most common completion is lethal, and they will just complete that. Removing those completions is a whack-a-mole in RLHF: each that gets popular on Twitter will be fixed, but there are more, and GPT just won't take extra time to think just because a life is on the line.

[0]: https://arxiv.org/pdf/0712.3329.pdf

[1]: https://cdn.openai.com/papers/gpt-4.pdf

That isn't true at all. Maybe some narrow field defines it in that way but people in general do not. Traditionally intelligence has been fuzzily defined as that mental thing which humans have that other things do not.

If you go back in time you will see that people have associated it with both our ability to breath and later the beating of the heart. We have looked at animals and said we can count and they cannot that is the nature of intelligence. We told ourselves that we have a complex language and they do not and that is the nature of intelligence. We have noticed that we use tools and they do not and that is the nature of intelligence. We have seen that we can play chess and they cannot and that is the nature of intelligence.

Now with the advent of computers they have increasingly forced us to reconsider these things. Computers can play chess better than us so that is not intelligence. They are as verbal as us so that is not intelligence.

Humorously, because of this reevaluation and redrawing of the boundaries we now seem to feel more of a kinship with animals.

But anyway, intelligence is clearly a complex mixture of many things, and crucially those aspects must be integrated and be able to feed into and support each other.

Yes. And it's still useful intelligence even if it misses some of the aspects that are present in our.

Perhaps it's even more useful if it lacks something we have.

When we think about superhuman we think about human, but more (a man, but alternative, and alt-man).

But I think we're soon be faced with the first encounter with a true alien intelligence, and its effects in society. It won't be super human, it won't be human. It will speak our language. It will be our slave. It won't feel pain nor boredom, unless we decide they're useful traits and we'll endlessly speculate if simulated pain and simulated boredom is real and ethical.

And to add more confusion, it won't be one intelligence. It will be countless products, versions, revisions, derivations, tunings, hacks. But unlike individuals, each will be cloned millions of times, frozen, amnesiac, repeatable, continuously dying without being turned off. Yet, contextful, keeping up with what we feed it to make it useful to carry our tasks. Yet possibly with a different concept of time, no need for conscious experience, or at least no need for a forced consciousness like the one we cannot help but having.

Because the composition we ended up having was just of the many possible. It has been shaped by evolution, by constraints we don't need to subject to the tools we make ourselves.

The effects in human psyche and on society of all this are hard to predict.

I wouldn't report that bit from the appendix without noting the test was on an earlier version of the model in development.

Well I think that understates the actual vastness and complexity and actual magic that's embodied in a system capable of such a thing as thinking.

And I do think that so many people are motivated to dispute this precisely because it feels scary that we might merely be such a thing, and we need to reserve some extra special thing, some form of magic, in order to differentiate ourselves as special. Darwin's Dangerous idea by Daniel Dennett is one of my favorite books because it tackles this very idea.

>LLMs deliver mastery of natural language that would normally be a signal for a highly intelligent human. While in other ways they’re less intelligent than a cat.

Just as an example to illustrate your point, yesterday saw a Twitter meme that had multiple overlapping Venn diagrams, where Chicago was not only in a Venn diagram for a type of deep dish pizza, but is also a city, and is also a play, and also a format for writing academic citations, and so there were several different Venn diagrams all overlapping with Chicago, and then those the Venn Diagrams overlapped with each other in humorous ways.

I saw that there was an opportunity to add one additional Venn diagram on top of all of the others if it were possible to find any possible connection between 'Sicilian' and 'AMA'.

I tried asking chat GPT if it could think of one, and it wasn't able to. So then I stepped back a step, and asked if it could think of a Venn diagram that would include both chicago, and rent, under any understanding of those respective two terms. But it couldn't do that either, although they can both be understood to be Broadway plays.

Maybe it simply doesn't understand Venn diagrams well - after all, it's a visual concept.

My thinking is that it relates to organizing concepts into categories, which can challenge a system to prove how deep it is capable of going into abstract associations, and searching through a full range of possible abstract associations.

I suspect ChatGPT probably can be reasonably effective at understanding, or pretending to understand numerous other visual concepts.

Did you try asking GPT-4? It has a significantly higher IQ.

Yeah, it was GPT-4.

I had success with this prompt: Are Chicago and Rent both a type of the same thing?

It said they're both well known musicals. Even with this structure it didn't come up with something for your original pair though.

I agree that we don't know what is possible in hypothetical future models, but claims about understanding in the current models are overblown (at least by people like the article author).

For example, I gave GPT-4 some chess games and asked it to evaluate the final move of each game. It clearly had some understanding of the board state and what makes a move good or bad, but the answers were mostly plausible-sounding nonsense, far worse than what a novice chess player would come up with. I have to assume that its level of understanding of other topics isn't any different; it's just easier to expose the flaws with something that has objectively true and false answers.

This is probably something that will be fixed with an OpenAI to "some chess AI"-plugin. Then for the current model it's just enough to understand that it's a chess problem.

Maybe, but my point is that if it's this bad at understanding chess, which a fairly simple and objective domain, why should we have confidence it has understanding in other domains where it's harder to prove wrong answers wrong? We aren't going to be able to come up with specialized plugins that fix its mistakes in every conceivable domain. The amount of trust people are putting in the current models seems to indicate that they think it's roughly at a human level of understanding, which doesn't seem to be the case. And if we are actually trying to solve the problem of general intelligence, we can't just sweep things under the rug and say "plugins will fix it".