Nature created humans to understand nature. We created GPT4 to understand ourselves.

A mirror of ourselves*

That's beautiful until you think about it.

Humans so far have done a great job at destroying nature faster than any other kind could.

And GPT4 was created for profit.

Humans are the universe asking who made it.

Yes, or put a bit more elegantly, 'The cosmos is within us. We are made of star-stuff. We are a way for the universe to know itself.' — Carl Sagan

Yes, "all serve "gpt4!" Get a grip, pretty please.

There is no evidence that intelligence runs on neurons. Yes, there are neurons in brains, but there's also lots of other stuff in there too. And there are creatures that exhibit intelligent properties even though they have hardly any neurons at all. (An individual ant has only something like 250000 neurons, and yet they're the only creatures beside humans that managed to create a civilization.)

This is not a good take. Yes there is a lot more going on in brains than just neuronal activity, we don’t understand most of it. But understanding neurons and their connections is necessary (but not sufficient) to understanding what we consider intelligence. Also, 250k is a lot of neurons! Individual ants, as well as fruit flies which have even fewer neurons, show behavior we may consider intelligent. Source: I am not a scientist, but I work in neuroscience research

What's the argument that understanding neurons is necessary?

Perhaps intelligence is like a black box input to our bodies (call it the "soul", even though this isn't testable and therefore not a hypothesis). The mind therefore wouldn't play any more of a role in intelligence than the eye. And I'm not sure people would say the eye is necessary for understanding intelligence.

Now, I'm not really in a position to argue for such a thing, even if I believe it, but I'm curious what argument you might have against it.

The other comments have pretty much covered it. We can pretty clearly demonstrate that neurons in general are important to behavior (brain damage, etc) and we even have some understanding about specific neurons or populations/circuits of neurons and their relation to specific behaviors (Grid cells are a cool example). And this work is all ongoing, but we're also starting to relate the connectivity of networks of neurons to their function and role in information processing. Recently the first full connectome of a larval fruit fly was published - stay tuned for the first full adult connectome from our lab ;)

Again, IANA neuroscientist, but this is my understanding from the literature and conversations with the scientists I work with.

Why would you doubt neurons play a roll in intelligence when we've seen so much success in emulating human intelligence with artificial neural networks? It might have been an interesting argument 20 years ago. It's just silly now.

> It might have been an interesting argument 20 years ago. It’s just silly now.

Is it?

These networks are capable of copying something, yes. Do we have a good understanding of what that is?

Not really, no. At least I don’t. I’m sure lots of people have a much better understanding than I do, but I think its hard to know exactly whats going on.

People dismiss the stochastic parrot argument because of how impressive big neural nets are, but it doesn’t really invalidate that argument. Is a very, very, very good parrot that learns from everyone at once doing basically the same as what we do? I’d argue no, at least not fully. It’s absorbed aspects of us extremely well/is a very weird, sophisticated mirror, yes, and is copying something somehow, probably in a way reminiscent of how we copy. Is it basically the same as what we’re doing when we think? Partially? Fully? Not at all?

A typical engineer would say “good enough”. That type of response is valuable in a lot of contexts, but I think the willingness to apply it to these models is pretty reckless, even if it’s impossible to easily “prove” why.

To be clear on the exact statement you made, I think you’re right/it’s pretty clear neurons play some very important role/potentially capture a ton of what we consider intelligence, but I don’t think anyone really knows what exactly is being captured/what amount of thought and experience they’re responsible for.

That person's argument is borderline insane to me - a severe lack of knowing what is unknown, a reverence of current best-models (regards modern science, including neurology - yet, open minded investigations beyond are also a requisite here.) And the pompousness is what truly boggles my mind ("Its silly to believe this, now.) A look in the mirror would suffice to say the least...

Anyway, thank you for a great answer and conversation throughout this thread.

Regards neural networks, parroting and the emulation of intelligence (or the difference between an emulation and the "real thing"):

Well, somewhat like you say, we cannot propose a valid comparison from one to the other without an understanding of one (consciousness) or both. It's fascinating that there are some open, valid and pressing questions about what / how the output of this new wave of software is concretized (from foundational, semi-statistical algorithms in this case.)

Yes, I do agree neurons have something to do with the "final output". But this is a subsection of the problem - organic neurons is-an/are order(s) of magnitude in complexity beyond what the tricky "parrot" is up to. Moreso, these components perform very different functionally - the known functions of the neuron compared to ANN, backprop etc. The entire stack.)

P.S: One interesting theory I like to simulate and/or entertain is that every organic cell in the body has something to do with the final output of consciousness.

Thank you, appreciate the compliment.

And yeah, are definitely a lot of open questions related to all of this. Love how its brought so many deep questions into focus.

Your theory makes sense in an evolutionary context. It is possible that all cells and organisms have some general intelligence. Humans do not have the ability to recognize this because evolutionarily it was only helpful to recognize intelligence when it could pose a threat to us. And the biggest threat to us in general was other humans as we are tribal animals. So we don't see it, we only see specialized intelligence that historically posed a threat to us.

It would explain why most "experts" didn't see GTP-4's abilities coming. Many of them expected that it would take a major algorithm or technology improvement to do "real intelligent" things, because they fundamentally misunderstood intelligence.

Please read the comment I was responding to. I was addressing the suggestion that perhaps the brain is as relevant to intelligence as the eye. Cognitive Neuroscience has been a thriving field for nearly half a century now. I didn't say we have it all figured out, just that it's obvious neurons are a piece of the puzzle.

If anything the experience with artificial neural networks argues the opposite - biological neurons are quite a bit different than the "neurons" of ANNs, and backpropagation is not something that exists biologically.

Brain damage by physical trauma, disease, oxygen deprivation, etc. has dramatic and often permanent effects on the mind.

The effect of drugs (including alcohol) on the mind. Of note is anesthesia which can reliably and reversibly stop internal experience in the mind.

For a non-physical soul to hold our mind we would expect significant divergence from the above. Out of body experiences and similar are indistinguishable from dreams/hallucinations when tested against external reality (remote viewing and the like).

“For a non-physical soul to hold our mind we would expect significant divergence from the above.”

This sounds like it assumes a physical mind could access a non-physical soul. All we probably know is that we have to be using an intact mind to use free will.

> Brain damage by physical trauma, disease, oxygen deprivation, etc. has dramatic and often permanent effects on the mind.

That's not a completely watertight argument.

Consider a traditional FM/AM radio. You can modify it, damage it, and get notable changes to its behaviour...

> Of note is anesthesia which can reliably and reversibly stop internal experience in the mind

...turn it off and on again...

> For a non-physical soul to hold our mind we would expect significant divergence from the above.

... yet concluding that all the noises produced from the radio are purely internal, mechanical and physical would be the wrong conclusion.

(I'm not arguing that the human brain/mind is anything like analogous to a radio, just pointing out the limits of this approach.)

I mean, if we're really going to go there, who's to say that a large enough LLM doesn't automatically receive a soul simply because that's one of the fundamental laws of the universe as decreed by the Creator?

Going where? I wasn't arguing for the existence of a soul.

Although, sure, if we could somehow manage to determine that souls did exist then presumably an AI model as capable as a human would also be eligible for one.

You can actually hypothesize that a soul exists and that intelligence is non-material, its just that your tests would quickly disprove that hypothesis - crude physical, mechanical modifications to the brain cause changes to intellect and character. If your hypothesis was correct you would not expect to see changes like that at all.

Some people think that neurons specifically aren't necessary for understanding intelligence but in the same way that understanding transistors isn't necessary to understand computers, that neurons comprise the units that more readily explain intelligence.

> You can actually hypothesize that a soul exists and that intelligence is non-material, its just that your tests would quickly disprove that hypothesis - crude physical, mechanical modifications to the brain cause changes to intellect and character. If your hypothesis was correct you would not expect to see changes like that at all.

That’s not necessarily a disproof. It’s also not necessarily reasonable to conflate what we call “the soul” with intelligence.

This is entering the world of philosophy, metaphysics and religion and leaving the world of science.

The modern instinct is to simply call bullshit on anything which cannot be materially verified, which is in many ways a very wise thing to do. But it’s worth leaving a door open for weirdness, because apart from very limited kinds of mathematical truth (maybe), I think everything we’ve ever thought to be true has had deeper layers revealed to us than we could have previously imagined.

Consider the reported experience of people who’ve had strokes and lost their ability to speak, and then later regained that ability through therapy. They report experiencing their own thoughts and wanting to speak, but something goes wrong/they can’t translate that into a physical manifestation of their inner state.

Emotional regulation, personality, memory, processing speed, etc… are those really that different from speech? Are they really the essence of who we are, or are they a bridge to the physical world manifest within our bodies?

We can’t reverse most brain damage, so it’s usually not possible to ask a person what their experience of a damaged state is like in comparison to an improved state. We do have a rough, strange kind of comparison in thinking about our younger selves, though. We were all previously pre memory, drooling, poorly regulated babies (and before that, fetuses with no real perception at all). Is it right to say you didn’t have a soul when you were 3 weeks? A year? Two years? When exactly does “you” begin? I can’t remember who I was when I was when I was 2 months with any clarity at all, and you could certainly put different babies in doctored videos and I wouldn’t be able to tell what was me/make up stories and I’d probably just absorb them. But I’m still me and am that 2 month old, much later in time. Whatever I’m experiencing has a weird kind of continuity. Is that encoded in the brain, even though I can’t remember it? Almost definitely, yeah. Is that all of what that experience of continuity is, and where that sense is coming from? I’ve got no idea. I certainly feel deeper. Remember that we all are not living in the real world, we’re all living in our conscious perception. The notion that we can see all of it within a conscious mirror is a pretty bold claim. We can see a lot of it and damage the bridge/poke ourselves in the eyes with icepicks and whatnot, and that does stuff, but what exactly is it doing? Can we really know?

Intuitively most people would say they were still themselves when they were babies despite the lack of physical development of the brain. Whatever is constructing that continuous experience of self is not memory, because that’s not always there, not intelligence, because that’s not always there, not personality, because that’s not always there… it’s weird.

I think it’s important to remember that. Whenever people think they have human beings fully figured out down to the last mechanical detail and have sufficient understanding to declare who does and doesn’t have a soul and what that means in physical terms, bad things tend to happen. And that goes beyond a statement to be cautious about this kind of stuff purely out of moral hazard; the continual hazard is always as empirical as it is moral. We can never really know what we are. Our perceptual limitations may prove assumptions we make about what we are to be terribly, terribly wrong, despite what seems like certainty.

I’m here playing devil’s advocate - this test doesn’t work. Here are some related thought experiments.

Suppose a soul is an immaterial source of intelligence, but it controls the body via machine-like material hardware such as neurons.

Or an alternative, suppose there is a soul inside your body “watching” the sensory activations within your brain like a movie. The brain and body create the movie & have some intelligence, but other important properties of the consciousness are bound to this observer entity.

In both these cases, the test just shows that if you damage the hardware, you can no longer observe intelligence because you’ve broken the end-to-end flow of the machine.

Its fine if you are playing or supposing in seriousness but with good humor, it doesn't really change how anyone else should interact with you :)

But yes, supposing that then you would expect to only see damages that correspond such as different forms of paralysis or other purely mechanical damages, not things that change the interior perspective.

Otherwise you start postulating the existence of a thing whose sole justification is your desire for the existence of that thing, which is natural when you start questioning beliefs and kick out all the supports without meaning to.

I think this is what Bertrand Russel's teapot was meant to ellucidate.

If you really want to present ants as a civilization, I don't think a single ant is a meaningful unit of that civilization comparable to a single human. A colony, perhaps - but then that's a lot more neurons, just distributed.

A single human will die permanently, alone, miserable, and unproductive. The comparison is apt.

A single human can survive on their own; there are many historical examples of that. A detached body part, on the other hand, cannot; but it also cannot feel miserable etc. A single ant is more like a body part of the colony in that sense.

If by "survive" you mean "age and die leaving nothing behind", then sure.

But the same is true for an ant.

No, it is not true for an ant. A solitary worker ant will not die from old age. Many species will literally starve to death within a few days because they cannot properly digest their food without trophallaxis.

You're right, it seems ants need each other to digest food, though scientists don't yet know why exactly.

From the New Yorker [1]:

> The researchers aren’t yet sure exactly what causes the digestive slowdown. Maybe it’s stress. Maybe the fluid that ants exchange when they share food stimulates digestion. Maybe they die still looking for a relative to break bread with.

[1] https://www.newyorker.com/tech/annals-of-technology/lonely-a...

In what way a civilization?

I'll repost a comment via Reddit that I think makes this case [0]:

Ants have developed architecture, with plumbing, ventilation, nurseries for rearing the young, and paved thoroughfares. Ants practice agriculture, including animal husbandry. Ants have social stratification that differs from but is comparable to that of human cultures, with division of labor into worker, soldier, and other specialties that do not have a clear human analogy.

Ants enslave other ants. Ants interactively teach other ants, something few other animals do, among them humans. Ants have built "supercolonies" dwarfing any human city, stretching over 5,000 km in one place. And ants too have a complex culture of sorts, including rich languages based on pheromones.

Despite the radically different nature of our two civilizations, it is undeniable from an objective standpoint that this level of society has been achieved by ants.

[0]: https://www.reddit.com/r/unpopularopinion/comments/t2h1vs/an...

To be honest, this description is leaning heavily on the associations we have with individual words used. Ant "architecture" isn't like our architecture. Ant "plumbing" and "ventilation" have little in common with the kind of plumbing and ventilation we use in buildings. "Nurseries", "rearing the young", that's just stretching the analogy to the point of breaking. "Agriculture", "animal husbandry" - I don't even know how to comment on that. "Social stratification" is literally a chemical feedback loop - ant larvae can be influenced by certain pheromones to develop into different types of ants, which happen to emit pheromones suppressing development of larvae into more ants of that type. Etc.

I could go on and on. Point being, analogies are fun and sometimes illuminating, but they're just that. There's a vast difference in complexity between what ants do, and what humans do.

Nobody is saying that an ant might be the next Frank Lloyd Wright.

They're saying they accomplish incredible things for the size of their brain, which is absolutely and unequivocally true.

"Go to the ant, thou sluggard; consider her ways, and be wise".

What I'm saying is that, Bible parables notwithstanding, it's not the individual ant that achieves these incredible things. The bulk of computational/cognitive work is done by the colony as a system. This means that there's little sense in comparing brainpower of an ant with that of a human. A more informative comparison is that between an ant colony and human society - here, humans may come out badly, but that's arguably because our societies are overcomplicated in order to compensate for individual humans having too much brainpower :).

> There's a vast difference in complexity between what ants do, and what humans do.

Interesting parallell with intelligence/sentience/sapience. Despite the means, isn't the end result what you have to judge? The end result looks like a rudimentary civilization. How much back in time would we have to go back to find more sophistication in ant societies than humans?

> Despite the means, isn't the end result what you have to judge? The end result looks like a rudimentary civilization.

I'm indeed talking about the ends. The operative phrase here is "it looks like". It looks like a civilization, there are even some structural similarities to be found - but it is not a civilization, anymore than a Lego brick with a computer sticker on it is a PC.

This is not to deny that ants are awesome and very much worth studying - my point is to avoid following flawed analogies too far. "Ant colonies look like rudimentary civilization, and civilization is/does X, therefore..." is going to be a nonsense statements for most of possible values of X.

You could say the same in reverse. Humans can't lift fifty times their own weight. Humans can't communicate in real-time with pheromones alone. Most humans do not know how to build their own home. An ant might well consider us backwards, not advanced.

> Humans can't lift fifty times their own weight.

Ants can do it as a consequence of their size. Relative lifting strength drops fast with increased size. Conversely, an ant scaled to human size would collapse under its own weight and cook itself to death - waste heat generation scales with volume (~ size³), while waste heat rejection scales with surface area (~ size²).

And it ain't a cognitive achievement anyway.

> Humans can't communicate in real-time with pheromones alone.

Yes. Because it's not as useful at our scale, nor is it real-time - chemical communication works better for small organisms and small volumes of living, as the travel speed and dissipation rate of pheromones is independent of organisms emitting them. Meanwhile, we have multiple ways of communicating real-time, some of which work at light speed which is the definition of "real time" (light speed in vacuum being the speed of causality itself).

> Most humans do not know how to build their own home.

Neither do ants.

> An ant might well consider us backwards, not advanced.

An ant can't consider us anything. The point I'm trying to get across is, just because the ant colony is capable of surprisingly high sophistication, doesn't mean the individual ants are.

As a counterpoint that's actually cognitive in nature: AFAIK individuals of many (most?) ant species can be easily tricked into following each other in a circle, and they will continue to do so until they start dying from starvation. This is because the ant isn't making a decision following a complex thought process - it's executing a simple algorithm, that works well in nature because nature is chaotic enough that stable ant circles are unlikely to form (and when they do, they're unlikely to be left undisturbed for long).

> "Agriculture", "animal husbandry" - I don't even know how to comment on that.

To give some examples of ant agriculture and animal husbandry, they cut and feed leaves to a fungus they farm for food. The fungus even communicates to the ants to switch to a different plant when the plant produces toxins in defense to cutting.

Ants herd aphids, protecting them from predators (ladybugs), secrete pheromones that direct and pacify them, and massage their abdomens to milk sugar dew.

I think you can call ant societies civilizations, but the same time you can call a multi cellular organism a civilization, too. Usually, those also come from the same genetic seed similar to (most) ant colonies. But more importantly, you have various types of cooperation and specialization in multi cellular life.

Do termites count as well then? Bees? I'm not being facetious, but these insects seem similar.

Perhaps an important point here is that ants pass the mirror test, which, to my knowledge, no other insects (that we know of) do.

What else would intelligence run on?

Every cell in our body, and every bacterium living in a body (e.g. gut flora), contribute to our intelligence. It looks plausible (to me) that there's one "top cell" among them that represents the "person", others just contributing via layered signals, but whether this "top cell" is a neuron or another kind of cell is unknown.

Ah, so you're not studied in neuroscience with some revolutionary theory. A shame.

Maybe the neurons are the hardware layer. The software is represented by the electronic activity. There is a good video https://youtu.be/XheAMrS8Q1c about this topic.

> There is no evidence that intelligence runs on neurons.

1. Neurons connect all our senses and all our muscles.

2. Neurons are the definitive difference between the brain and the rest of the body. There is “other stuff” in the brain, but it’s not so different from the “other stuff” that’s in your rear end.

Don’t underestimate what a neuron can do. A single artificial neuron can fit a logistic regression model. A quarter of a million is on then scale of some our our largest AI, and biological neurons are far more connected than ANN. An ant quite likely has a more powerful brain than GPT-4.

> 2. Neurons are the definitive difference between the brain and the rest of the body. There is “other stuff” in the brain, but it’s not so different from the “other stuff” that’s in your rear end.

Our digestive systems appear to be important to our behaviour, though. Some recent work in mice showed that if colonised with bacteria from faeces of humans with autism, the mice would begin to show autistic behaviours.

So, not sure your argument here is especially strong.

The role that digestive systems potentially play for our behavior is via neurotransmitters, so in the end it is the neurons, that are responsible.

If the study you mention show something else, I would actually be very interested in a source.

There is no evidence that any of those are emergent properties. It’s no more or less logical than asserting they were placed there by a creator.

There is evidence that people believe that GTP-4 is intelligent since it can solve things like the SATs. But if you start taking away weights one by one at some point those same people will say it isn't intelligent. A NN with 3 weights cannot solve any problems that humans believe requires intelligence. So where did it come from? I don't know, but it clearly emerged as the NN got bigger.

It's possible that the comment you're responding to is referring to the new paper out of Stanford/Google on emergence as a 'mirage'. The gist of it is that the shape of the curve for 'metric' vs 'params' is dependent upon the type of metric. For classification it has a certain shape, for accuracy it has another, etc. The authors claim the observation that these curves are determined by the choice of metric means that it isn't true emergence, but rather is due to the way in which GPTs are typically trained to predict the next token (and then beam search or typical sampling is often applied afterwards, etc).

The paper is somewhat new so I haven't done a proper review to know if it's solid work yet, but this may offer some context for some of the comments in this thread.