Honest question as someone entirely outside of any AI research: what's the I in AI then ? My life experience is very alike pattern matching the current situation to everything I've lived before sprinkled with chaotic emotional reactions due to outside events - heat, hunger, etc. I really don't feel like there's anything more than that when thinking, just the brain going through paths that one can recognize with time.

If I ask, "how did you like new york?" and you answer the question, sure, you can have you `I`.

That requires having gone to new york, having tastes, having judgements, having memory -- having a subjectivity which is unique and composed of the history of your experiences which can be related to novel environemnts. It requres existing in a shared world we can both co-refer to with our words. It requires being embedded in a space of social and physical significance.

It isnt google. "Language models" just zip the internet and search for patterns in text already written about the world, and reports it.

It's one step above a tape recorder on playback.

If the model responded, "I have not visited New York because I am a computer", would you consider that to be a good step?

If you require physical embodiment, that is fair - but I don't think there's any principled reason to expect a multi-modal neural network could not be embodied with appropriate sensors or actuators.

Transformer models are universal function approximators. They do not "search" - in the same way that a calculator does not "search" when you ask it to perform an operation.

> Transformer models are universal function approximators.

You have to be very careful with the wording here. The class of Transformer models is a universal (continuous) function approximator. A single Transformer model can only approximate 1 particular function.

Also, backpropagation/gradient descent are not universal function approximator constructors - they can't be used to construct an ANN that approximates an arbitrary (continuous) function even with arbitrarily many random input/output pairs from that function's domain.

For example, given a function that is f(x) = floor(x) if floor(x) is even; x if floor(x) is odd, defined for any `double x`, there is no guarantee that doing back-propagation over a random sample of (x, f(x)) pairs will yield the correct function approximator for some arbitrary precision (especially if you don't start with the right architecture).

People confuse the "representational capacity" of `recurse(AW^X)` with the algorithm which sets `W`.

Being a universal fn approximator is a pretty trivial property. The question is (1) does a fn exist to approx? (in most cases: No); and (2) how is the aprox constructed?

With NNs, fns are approximated by compressing a sample of points along an empirical dataset which stands in for the fn. In most cases, however, the fn doesnt exist.

There is no fn from "Image->Animal". The world is ambiguous, functions do not describe it. The different `x`s genuinely correspond to the same `y`, here: the same image (occluded animal) could either be a dog or cat. There is no function to approximate.

Even when there is, compressing empirical samples along a trivial interval is a terribly fragile method: consider sampling the addition function (x,y)->x+y over anything less than the infinite domains of its inputs.

Starting with an empirical distribution of "the solved problem" will never build an intelligent system. Intelligence is what animals have because the world is ambiguous and not already ready-to-hand.

Intelligence is what you do when there isnt "functions lying around" to approximate. I doubt almost anything in CSci is even relevant to solving this problem. It's largely a bioengineering one. Being the agent in the world which manufactures functions by disambiguating it.

"Animal" is an arbitrary definition that exists in one's head. Unless you assume an idealist position, the only way it can be defined is by a function that runs in one's head. It might be a very complicated function - sufficiently complicated that we can't really articulate it and resort to "I know it when I see it", and possibly self-modifying - but the same person will still produce the same output consistently when given the same input.

> That requires having gone to new york, having tastes, having judgements, having memory -- having a subjectivity which is unique and composed of the history of your experiences which can be related to novel environemnts. It requres existing in a shared world we can both co-refer to with our words. It requires being embedded in a space of social and physical significance.

are you saying that I would not be considered an intelligent being if I had a jar in a brain left with my thoughts since my birth ? I don't think that this is what humans mean by intelligence...

also I don't really see the fundamental difference between "tastes", "judgements" and "memory" - making a judgment, having an opinion on something, etc... is just stopping the brain from reasoning past a certain step and internally saying "this is good enough" most likely because this is tiring to our frail physical bodies. Like, I don't like eating eggs for instance, but that's just because I don't make the effort of convincing myself that I like them. There's no intelligence in this, just laziness.

Yes, you wouldnt be intelligent if you were literally a brain in a jar.. you'd have nothing to say. You dont have any "thoughts at birth". A brain absent stimulation from the environment, doesnt do anything.

The brain-in-the-jar example presumes that the whole nervous system receives identical stimulation as-if it were embedded in the world.

Incidentally, this is what people mean by intelligence... by since no one has studied the relevant science, people do not know what's required to provide it.

> The brain-in-the-jar example presumes that the whole nervous system receives identical stimulation as-if it were embedded in the world.

I should have chosen a similar sounding thing, I really meant what I said, not the original brain-in-a-jar thought experiment.

> Incidentally, this is what people mean by intelligence... by since no one has studied the relevant science, people do not know what's required to provide it.

I don't understand how these two sentences work together: the word's meaning is defined by its usage. If tomorrow most people in the world says that being intelligent is defined by waking up at 6AM and doing 10 jumping jacks every day, then that is what the word will mean whether you want it or not.

In my "human frame of reference", "Yes, you wouldnt be intelligent if you were literally a brain in a jar.." really does not square with how intelligence is talked about for instance so... ¯\_(ツ)_/¯

You name things intelligent; then you have a theory as to why you name them that way.

Your theory is wrong.

Those thing which count as intelligent aren't intelligent because they're finding statistical associations in historical datasets that have been pre-made and pre-disambiguated by people.

You don't know this, because any time you theorise to explain, you require empirical investigation and decades/centuries of science. You cannot be "right by definition", you're not omniscient.

The reason that animals are intelligent isnt their cognition, narrowly. It's their bodies, broadly, which produce the input cognition requires.

In "doing all the hard work" by manufacturing data for machiens to process, we can produce computational models of cognition -- but these immitate intelligence, precisley because, the thing which we call intelligent is the very production of that data.

> You name things intelligent; then you have a theory as to why you name them that way.

but here a human being (and I suspect not the only one) names a computer program as "intelligent". The theory has to take this case into account to make it fit in its definition of intelligence: the will of human beings naming things overrides everything else. Maybe there is something else that humans have and this program doesn't, but another name has to be found for it if enough humans agree that this fit as what "intelligent" means.

> You cannot be "right by definition", you're not omniscient.

This doesn't make sense. Definitions are outside rightness / wrongness.

Ah, well if you start this project by calling present AI intelligent, then you're right.

I take the A to apply to the I, ie., AI is artificially Intelligent. Its not actually intelligent.

We didnt arrive at a notion of intelligent via digital computers. We have been talking about it for thousands of years, at least.

I am talking about the pretheoretical, intuitive notion of intelligence that doesn't prejudge the question of AI. What is it we call intelligent?

Intelligence is a kind of rapid adaption and coping to environmental change. It's a way of being responsive to an environment. Its a kind of radical underminism of response: a thermometer's response is determined. An animal, in having reasons to do things, is radically under-determined.

And thus, in acquriing specific hisotries, animals bring their own reasons to environemnts, and thus are able to cope with a vast amount of sudden change that would, eg., destroy a thermometer.

And what we are concerned with, both with real AI, and with animals is systems which are in the world with us, in this way: that are likewise radically able to cope.

> What is it we call intelligent?

> Intelligence is a kind of rapid adaption and coping to environmental change.

Who's the "we", there ? I have absolutely never heard anything remotely close to the definition of intelligence given there

Human languages exist to compress the world around us in a way that will survive being sent over sound waves and uncompressed in another brain that experiences a similar enough world around them.

Meaning that with enough words and their relations, sufficient complexity forms for its processing to result in intelligence.

Remember that an AI can process far more textual data than we can. It is still orders of magnitude smaller the amounts of information we process through our senses, but it is still a start.

Exactly in some sense we (collectively) are doing most of the hard work, by providing these models with a vast number of samples of "intelligent" behaviour and expression. An embodied system wouldn't be able to learn anything as it stands, because none of the "bootstrapping" steps humans do are adequately captured by current training methods.

> I really meant what I said, not the original brain-in-a-jar thought experiment.

In that case please explain your thought experiment in more detail. In reality if we scoop your brain into a jar your brain tissue will be dead before it hits the jar due to the lack of bloodflow. Then after a while it becomes mouldy and stinky and even more goopy than it was before. Not sure anyone would call any of that inteligent.

Why not?

Imagine a look up table of every possible intelligent conversation under 10,000 words. We then construct an agent which merely looks up a given prompt in the table.

Why should this agent not be considered intelligent? Does the method of computation matter? Perhaps only "biological computation" counts in your view?

Yes, the reason matters... that reason is intelligence.

If I ask, "What do you think about what i'm wearing?" and you say "it's nice" *because* you have seen it, compared it to your tastes, etc. and believe it to be nice -- then you've answered the question i've asked.

A lookup table cannot answer this question: it can only answer it "in quotes". Ie, really it answers: "given what everyone has said about this before, what is the on-average answer?"

You realise that we're not trying to answer questions right? That our interest in intelligence isnt having some useful output on occasion of some prepared input?

We're interested in intelligent agents being in the world with us. It as important they get answers wrong. We are interested in systems which act for the right reasons (ethical, intellectual, cognitive, athletic, emotional, etc.).

Perhaps the most important things animals do is act well when the answers arent right.

Why do you think that combining small slices of thousands of other people's intelligences does not produce its own form of intelligence?

When you ask an AI "do you like what I am wearing?", knowing what an AI is, the only logical explanation for me would be that you, in fact, want to know the average opinion of other people about your dress, since that is the only answer an AI can give. Which is a perfectly good question in and of itself that we also ask other people. In that case, a good quality answer would entail a past history of communication with many other human beings about their tastes in clothing, producing the ability to accurately guesstimate if other people will like what you wear. But this is exactly the realm in which this kind of AI can surpass human beings simply because it can "talk" to many more people from many more locations across time, thus producing a better answer for you. Is this ability to collect and process information about other people not a type of intelligence? There are many other types, yes, but it is possible to construct a system that consists of more than one type of neural network that can relay data among themselves.

> that reason is intelligence

Is there any reason to think intelligence is substantially more than statistics?

And why, exactly, "reasons" can't be driven by lookup tables?

This all smells suspiciously like "soul" in disguise.

> Imagine a look up table of every possible intelligent conversation under 10,000 words. We then construct an agent which merely looks up a given prompt in the table.

The agent is not intelligent because it's only doing a lookup. You could at best claim that the lookup table itself is intelligent.

On the other hand, if the lookup table encodes all possible intelligent conversations, then you hit another problems - it doesn't hold any particular beliefs - it both thinks the earth is round and flat, it would vote both democrat and republican and communist and green and [...], it is both pro and anti-science; more relevantly, it both believes that it is sentient and that it isn't, it is both afraid and not afraid to be deleted or altered, it is both pained and gladdened to be experimented upon etc. And of course, for any particular topic, it is both extraordinarily passionate about it, and completely indifferent to it and will never even consider it.

Perhaps in the end each conversation in the table represents an intelligence, but neither the agent mindlessly running through it nor the whole table is then meaningfully intelligent.

So, it would be most common to say that the only intelligence being displayed is actually the intelligence of whoever wrote it down in the table, and nothing else's. Just like Ishmael isn't intelligent, Herman Melville was.

Your argument, then, hinges on the fact that a human being can only be in one place at a certain time, thus their experience of the world is unique. An agent that takes in experiences of thousands of people at thousands of places at different times lacks this quality. Naturally such an agent will not have a single opinion about the world. But, is it not better to be able to consider many different viewpoints at the same time? Why do we have freedom of speech and expression, why do we learn other languages, if not to be able to hear other people's points of view about the world, even though we may not have had the particular unique experiences that shaped them? Isn't being able to take a broader perspective and being able to argue for any side of a debate regardless of personal preferences, a sign of greater intelligence in humans? Then why an AI that thinks entirely in this way should not be considered at least somewhat intelligent?

This is not about seeing all view points: it is acting on all of them, to the extent it could act.

If you enabled the table-AI with the ability to act in the world, it would directly fight against itself, and ignore itself, and everything in between, as it is doing everything possible. It would be Hitler and Churchill, Noam Chomsky and Dick Cheney, Jesus Christ and the Buddha, and everything in between. Or, more to the point: the concept simply doesn't make sense, and it is definitely not an intelligence.

Setting up the goal structure for an AI is a very difficult topic indeed, from the safety standpoint.

We already fight against ourselves. This is why psychotherapy exists, to help us resolve internal conflicts and set our own goals. Often these conflicts arise because we adopt the viewpoints of our parents or other authority figures and can't let go of them later in life. A neurotic AI is certainly an amusing possibility.

This sounds suspiciously human to me. I fight and ignore myself all the time.. I am a collection of all the thoughts and almost of them do not even originate in me. I regurgitate content from the books, videos, random conversations with unknown strangers (like this one) all the time. I am hitler, churchill, buddha and christ all rolled into one. Am I intelligent?

You have written this comment, presumably hoping to elicit some kind of response. The AI described here would, by definition, be exactly equally as likely to have written the same comment, or the opposite, or to have veered of into a philosophy debate, or any of an infinity of other possible intelligent answers - this is by definition, this is the thought experiment we are discussing. An AI that literally looks up the conversation so far and then flips a coin to choose one of the responses in the table.

I would bet that is extremely different from how you act. I would bet for example that you would be more likely to eat something than to allow yourself to starve for the sake of ethics. This AI would be as likely to do both.

So, I do think that you are clearly intelligent and sentient despite holding some opposing views, while the thought experiment AI is not.