I'm becoming convinced that these algorithms are huge steps towards AGI simply because AGI might end up being a collection of many of these domain-specific networks with a network sitting above them who's only role is to interrogate the sub networks for solutions to the problem at hand, and discriminate which solution(s) are most worth trying, simulating those and then picking one out and executing it in the real world. That seems to me to be pretty close to what we as humans do.

You just need a neural network that knows how to interrogate other neural networks, and a neural network that knows how to find new datasets and train new neural networks and you potentially have something very scary right?

Maybe a third neural network that knows how to purchase cloud compute?

>Maybe a third neural network that knows how to purchase cloud compute?

Amazon has entered the chat

Private angel investors watch from above all of the “clouds”

Cognitive scientists have been exploring this idea for a while, e.g. the Modularity of Mind Hypothesis. It would be very interesting to see if learnings from this field can be further applied to AI development.

the problem you run into then is how does it adapt to new domains? sure large models display some degree of generalization, but new tasks will certainly ask things the network couldn't get

I think that's valid, however, I think the modular approach would allow for a lot more cross domain flexibility because I suspect that a large subset of disparate tasks do actually decompose themselves into discrete subtasks that can be useful across domains.

Adding to that, if you can compose multiple sub networks together then you've really got something. You can build a lot of different buildings from bricks without needing to invent new kinds of brick basically.

For instance, think about the large number of domains that robust computer vision would be useful in. Then think about the fact that if the computer understands the 3D space around it, it can hand that model off to a network that does predictive physics simulation. Now you've got something that would be useful across a extremely wide range of domains.

The modular approach doesn't need to be enforced by strict boundaries. One network that starts with various components but can be remodeled and even merged together as it goes could adapt.

What we're discussing here is merely a starting state of something that'd have to rebuild itself.

Or maybe evolving modalities within one network is easier than putting it together.

Exactly (and cannot recommend Blindsight highly enough). Of course DALL-E is a Chinese Room. The most exciting and subversive idea in Blindsight is that consciousness is maladaptive, and the scramblers are _more_ fit for the lack of it. Long after neutral nets have surpassed our functional intelligence, we'll still be criticizing their ability to navel-gaze as well as humans.

Of course, DALL-E can only get away with not being introspective because it has reinforcement learning holding its hand. Bit harder to curate a clean training set without consciousness.

Personally, I suspect consciousness is intrinsically connected with whatever lets humans learn things by looking at and thinking about a single example.

I get what you're saying, but it's important to also consider how evolution provides for us as well by giving us neural structures already tuned for things like recognizing faces and recognizing danger. I do wonder how much realtime danger analysis has to do with consciousness.

I'd doubt it has much of a connection. Consciousness can do many things, but generally not quickly.

Hmm. I'd say subconscious might get you away from immediate danger, but it's often a conscious decision to anticipate future danger based on past events, and this feedback loop is sometimes quite small, such as deciding which path to take while being pursued.

I'm reminded of a meme...

>Giant robot pilot looking smug Caption: I am a genius!

Picture break: >same robot pilot now looking horrified. >Caption: Oh no!

I can tell you right frigging now, the average person is trained through education to build up confidence they know what they are doing, then over the next decade plus face-checked against reality to build an aversion/paranoia matrix.

The modern world is essentially orchestrated by people who build self-sustaining support structures of specialists. We've done a great job at building co-dependent specialists, but a terrible one at building generalists.

Reinforcement learning is the thing that integrates the network across time though. In one sense, I think it's doing at least part of the job that consciousness does: mediating between memory(past) and the present.

Yes, it's just that because it's an external process, DALL-E can't go meta on it like we can. So you won't ever get "appearance of consciousness" out of a process that depends on RL, imo, because you've located part of its cognition outside its brain and made it intermittent.

Consciousness is very much adaptive, though it requires a sense of the metaphysical / transcendent to behave in that way. Theologically speaking, God brings Life and the Devil brings Death. Sadly, our modern brave new world has taken "Gott ist Tot" as a triumphalist proclamation, and not the lamentation it was meant to be.

It's far too early to tell if consciousness is adaptive. Has it enabled humans to be wildly successful? In the short term, sure! But the long term is unclear.

E.g., look at how the technological progress it makes is at risk of destroying human civilization. A series of ecological boom/bust cycles could lead toward negative selection for consciousness. Or we might manage to wipe out ourselves or, say, most mammals, leading life on earth to start over as it did 65m years ago.

But even without that, it's not clear to me that consciousness will really win out. Look at the number of successful people who are not only painfully unreflective, but need to be to keep doing what they're doing. I could name a lot of people, but today's good example is Alex Jones, whose whole (very profitable) schtick is based on refusing to be fully conscious of what he's saying: https://popehat.substack.com/p/alex-jones-at-the-tower-of-ba...

And this is hardly a new idea. Vonnegut wrote a novel where humans end up evolving into something like a sea lion. The point being "all the sorrows of humankind were caused by 'the only true villain in my story: the oversized human brain'", an error evolution ends up remedying.

Hardly a new ideea indeed: https://en.wikipedia.org/wiki/Tree_of_the_knowledge_of_good_...

Edit: To be clear, I posit that consciousness is the organ that enables us to distinguish between Good and Evil.

I doubt that consciousness is an organ, but if it were, it's distinct from a moral sense, because we share that with animals. Here's a good book on the topic: https://www.hup.harvard.edu/catalog.php?isbn=9780674356610

That same professor has done a bunch more work on the topic, as have many others.

Frans de Waal has done great work.

Evolution is a strange phenomenon. I invite us to marvel at the transformation of terrestrial quadrupeds into majestic aquatic creatures, over eons: https://en.wikipedia.org/wiki/Evolution_of_cetaceans

Evolutionary speaking, cetaceans "share" the front limbs with quadrupeds. And yet there is a qualitatively distinct functional difference. Consider that moral consciousness, as present in humans, is functionally not quite the same as its biological precursor, the moral sense present in dogs or gorillas. And, of course, there are gradual changes along the evolutionary way.

Edit: "Organ", more precise "sensory organ", as in "the visual organ". Perhaps there is a better word here than "organ" here, before we get lost in the medical distinction between eye / retina / optic nerve / cortex / etc.

Sure. Nothing is the same. Every animal is slightly different. As humans trying to understand the world with 3 pounds of meat, we abstract and group so we have a chance of dealing with the overwhelming complexity.

But I stand by my point. A moral sense is something that provably exists in a lot of social animals. Is it different in each species? Yes. Is it different in each individual? Surely. But unless you're going to claim that all of those animals are conscious, you can't say that consciousness is foundational to a moral sense. Indeed, I think the much easier argument is that a moral sense is foundational to evolving consciousness.

Why we should be reduce human morality to what can be "provably" found in other social animals? There are some rudimentary similarities, but those similarities do not circumscribe the entirety of human moral experience, not by a large stretch.

You're now arguing against things I didn't say, so I think I'll leave this discussion here.

It's entirely unclear that no other animals exhibit consciousness.

Rudiments? Probably. Existential dread? Sainthood? Unlikely.

But Sontag suggests the use of Batailleist `powerful communication’ to modify and read consciousness. The subject is contextualised into a predialectic nationalism that includes language as a totality.

Hi there GPT-3 bot.

Edit. Don't really want to be unnecessarily flippant here. Perhaps this is a real person attempting to communicate. Sadly, I was unable to follow through, the text jumps wildly between seemingly unrelated subjects.

Nope, that was actually the Postmodern Generator.^[1] But nice try! ;) [1]: https://www.elsewhere.org/pomo/

Haha, <fist bump>.

The "Chinese room" argument, so far as I understand it, applies to any discreet computation process simulating consciousness.

The argument of the article is that DALL-E doesn't respond appropriately to a particular kind of input - two entities in some kind of spatial relationship (that it hasn't often seen). Dall-E's not extrapolating the three-D world but stretching a bunch 2-D images together with some heuristics. That works to create a lot of plausible images sure but it implies to this ability might not, say, be able to be useful for the manipulation of 3-D space.

So, given a "Chinese room" is just a computation, it's plausible that some Chinese room could handle 3-d image manipulation more effectively than this particular program.

Which is to say, "no, the criticism isn't this is a Chinese room, that is irrelevant".

The Chinese room argument itself isn't very compelling. Surely the constituent parts of the brain are fundamentally governed solely by physics, surely thought arises solely from the physical brain, and surely the constituent parts (and thus thought) could be described by a sufficiently complex discreet computation.

Are we not conscious?

The argument you make here is a reasonable one (IMHO) for the plausibility in principle of what Searle calls “strong AI”, but he claims that his “Chinese Room” argument proves that it must be mistaken. One can simply ignore him, but to refute him takes a little more effort.

It turns out that when one looks at the argument in detail, and in particular at Searle’s responses to various objections (such as the Systems and Virtual Mind replies), it is clear that he is essentially begging the question, and his ultimate argument, “a model is not the thing modeled”, is a non-sequitur.

The argument is essentially that there are no qualia of Chinese comprehension in an automaton or in any system that uses an equivalent algorithm, whether or not run by a human.

It's a sound argument to the extent that qualia clearly exist, but no one has any idea what they are, and even less of an idea how to (dis)prove that they exist in external entities.

It's the materialists who are begging the question, because their approach to qualia is "Well obviously qualia are something that just happens and so what?"

Unfortunately arguments based on "Well obviously..." have a habit of being embarrassingly unscientific.

And besides - written language skills are a poor indicator of human sentience. Human sentience relies at least much on empathy; emotional reading of body language, expression, and linguistic subtexts; shared introspection; awareness of social relationships and behavioural codes; contextual cues from the physical and social environment which define and illuminate relationships; and all kinds of other skills which humans perform effortlessly and machines... don't.

Turing Tests and game AI are fundamentally a nerd's view of human intelligence and interaction. They're so impoverished they're not remotely plausible.

So as long as DALL-E has no obvious qualia, it cannot be described as sentient. It has no introspection and no emotional responses, no subjective internal state (as opposed to mechanical objective state), and no way to communicate that state even if it existed.

And it also has no clue about 3D geometry. It doesn't know what a sphere, only what sphere-like shading looks like. Generally it knows the texture of everything and the geometry of nothing.

Essentially it's a style transfer engine connected to an image search system which performs keyword searches and smushes them together - a nice enough thing, but still light years from AGI, never mind sentience.

Searle’s argument is not about qualia; it is (as Searle himself has repeatedly stressed) about syntax, semantics and understanding. The argument simply does not consider what the room’s occupant feels.

Even if it were about qualia, calling the argument sound “to the extent that” we don’t know enough to tell whether its premises are correct would be a misuse of ‘sound’ and a rather blatant case of burden-shifting - effectively saying “so prove me wrong!” to skeptics.

Materialists can and do make question-begging claims, but that does not somehow cancel out Searle’s own question-begging (furthermore, somewhat ironically, Searle describes himself as a materialist!)

The soundness of the argument cannot be established by showing that current technology is far from being strong AI, as the argument claims much more than just that - it claims it to be impossible in principle. Anyone making such a claim has assumed a heavy burden that demands stronger arguments than you are making here.

>A model is not the thing modeled is a non-sequitur.

Have you ever used a map?

If not, I'd like you to get one and I want you to point out your home in every one. Can you use a magnifying glass, then look at the map hard enough to see yourself looking over a tinier map? Ad infinitum?

That is what Searle means. A map is a model of the world. The model of a world that is a map is not, in fact, in any way equivalent to or interchangeable with the thing it models. It is merely a distilled representation that provides a facsimile representative enough to be useful. So to would be any attempt at modeling consciousness.

This is an argument from the general to the specific which does not apply in this particular case, nor in many others like it. As a mind is plausibly an information process ocurring within the body, this generalization does not rule out an informational model of the physical processes of that body producing a mind.

If the argument you present here could be so easily generalized, it would work just as well for “proving” that a computer model of an Enigma cypher machine cannot encipher text.

I think a considerable subset of the people who do make use of the Chinese room argument also subscribe to some form of mind-body dualism, where consciousness does not or does not completely arise from physical processes.

I see it the other way round.

The Chinese Room and the brain of a Chinese-speaking person are completely different physical processes. Looked at on an atomic level, they have almost nothing in common. Mind-body dualists may or may not agree that the room is not "conscious" in the way a human is, but if consciousness is purely a material process, I can't see how the materialist can possibly conclude all the relevant properties of the completely dissimilar room and person are the same.

Those that would argue the Chinese Room is "conscious" in the same way as the Chinese person are essentially arguing that the dissimilarity of the physical processes is irrelevant: the "consciousness" of the Chinese person doesn't arise from molecules bouncing around their brain in very specific ways, but exists at some higher level of abstraction shared with the constituent molecules of pieces of paper with instructions written in English and outputs written in Chinese.

The idea our consciousness exists in some abstract sense which transcends the physics of the brain is not a new one of course. Historically we called such abstractions souls...

The obvious counterpoint is that if I followed your argument to absurdity, then I would also have to conclude that if I am conscious, then you can't be, because the atoms of our brains aren't arranged in precisely the same way. It clearly makes more sense from a monist point of view to consider consciousness an emergent property of complex systems, rather than one particular process.

To be more explicit, I'm saying I find it weird to hear so much about the Chinese room argument from a crowd of (presumably) materialists.

Of course! But if the universe is the result of all the quantum field interactions, what if there's a quantum field that, on its own, brings the consciousness interaction, and it manifests in ways that are computationally prohibitive for a process created from atomic-scale logic gates to replicate believably?

What if there's just no way to build consciousness from the building blocks that are within our reach?

Penrose thinks this (that the brain requires quantum computing), but it doesn't seem like anyone agrees with him or that it makes much sense. If I'm a quantum computer, why can't I do Shor's algorithm in my head?

Of course parts of regular computers involve "quantum stuff", like details of how transistors and hard drives work, but that doesn't mean they're magic.

I think we need to start thinking of these networks not as something we hope will someday simulate a brain, but as small building blocks that can be networked together in a variety of ways to tackle tasks. DALL-E may not understand 3D image manipulation, but if you chained the output from DALL-E into something like NeRF you could start to transform the image in 3D space. One can picture a collection of a few hundred of these networks, all with different things that they are good at working in concert to approach something that is very good at doing things generally. We're missing the layer on top that decides which algorithms to run and in what order.

fantastic book. Made me consider the question of whether consciousness exists at all or if it is just some hack by evolution to allow introspection.

I haven't found a definition of consciousness which is quantifiable or stands up to serious rigour. If it can't be measured and isn't necessary for intelligence, perhaps there is no magic cut-off between the likes of Dall-E and human intelligence. Perhaps the Chinese-room is as conscious as a human (and a brick)?

Also read the book due to comments here. Was very aligned with some of my interests in relationships between language, information, life/intelligence and metaphor <3

You might get a kick out of this paper (though some may find it's proposal a bit bleak, I think there's a way to integrate it without losing any of the sense of wonder of the experience of being alive :) )

It analogizes conscious experience to the a rainbow "which accompanies physical processes in the atmosphere but exerts no influence over them".

Chasing the Rainbow: The Non-conscious Nature of Being (2017) https://www.frontiersin.org/articles/10.3389/fpsyg.2017.0192...

> Though it is an end-product created by non-conscious executive systems, the personal narrative serves the powerful evolutionary function of enabling individuals to communicate (externally broadcast) the contents of internal broadcasting. This in turn allows recipients to generate potentially adaptive strategies, such as predicting the behavior of others and underlies the development of social and cultural structures, that promote species survival. Consequently, it is the capacity to communicate to others the contents of the personal narrative that confers an evolutionary advantage—not the experience of consciousness (personal awareness) itself.

So consciousness is more about what it subjectively feels like to be under pressure/influence to broadcast valuable internal signals to other (external) agents in our processes of life; aka other humans in the super-organism of humanity. I analogize it to what a cell "experiences" that compel it to release hormonal signals in a multicellular organism.

> Made me consider the question of whether consciousness exists at all or if it is just some hack by evolution to allow introspection.

Can't it be both? What's the difference? Evolution just responds to the environment, so a method of complex interaction with the environment like "consciousness" or "ever-polling situational awareness" seems like par for the course.

Evolution is the result of the environment not a response.

Giraffes didn't get a long neck because the food was out of reach, giraffes have a lock neck because the one without just died.

It's a response caused by survival. The environment is the stimuli and the genetics that are not eliminated are cultivated.

If I introduce an antibiotic into a culture of bacteria and they evolve resistance, then they appear to be responding to it on a collective level.

"Since bacteria are extremely numerous, random mutation of bacterial DNA generates a wide variety of genetic changes. Through mutation and selection, bacteria can develop defense mechanisms against antibiotics."

https://health.mo.gov/safety/antibioticresistance/generalinf...

Mutation happens all the time because cell replication isn't perfect, some mutation are irrelevant, some deadly, some bring better chance of survival.

It's not a response just the result. Or how does the bacteria know it's an antibiotic and not just water? It doesn't, water just isn't a evolutionary filter, antibiotics are.

It is a hack alright. A very beautiful hack, refined in hundreds of thousands of iterations, with a very high price.

> to allow introspection

Evolution doesn’t do things “to anything”. It repeats what works, and kills the rest. Our brains have allowed us to adapt to the changes in the environment better than the rest. Conscience came with the pack. It might not have an actual “purpose”- it could be an “appendix”.

My personal belief is that consciousness started as the self-preservation instinct that most animals have, and we developed introspection as a way to strengthen our ties to other members of our family or tribe. And then we “won” (for now)

> It is a hack alright. A very beautiful hack, refined in hundreds of thousands of iterations, with a very high price.

To what extent does this describe evolution? :-)

That is my description of evolution. The part that isn’t there is all negative: that it doesn’t (can’t) care about individuals. That it doesn’t (can’t) “understand” what is doing. That it doesn’t (can’t) “design”; it can only iterate. That it doesn’t (can’t) care that the intermediate result is inscrutable to us.

Mostly clarifications for people who anthropomorphize the process.

I think in these sort of topics we are often discussing two different meanings of consciousness: physical vs philosophical. The arbitrary mixing and matching of them is likely one the reasons for the lack of clarity. Physical consciousness is most easily identified by the notion that you're unconscious when sleeping. The other is the far more interesting philosophical consciousness.

Philosophical consciousness is what the oft misunderstood quote cogito ergo sum, I think therefore I am, was hitting on. Descartes was not saying that consciousness is defined by thinking. He was trying to identify what he could know was really real in this world. When one goes to sleep, the dreams we have can often be indistinguishable from a reality in themselves, until we awake and find it was all just a dream. So what makes one think this reality isn't simply one quite long and vivid dream from which we may one day awake?

But this wasn't an appeal to nihilism, the exact opposite. The one thing he could be certain of is that he, or some entity within him, was observing everything. And so, at the minimum, this entity must exist. And the presence of this entity is what I think many of us are discussing when we speak of consciousness. In contrast to physical consciousness, you are philosophically conscious even when sleeping.

Of course like you said philosophical consciousness cannot be proven or measured and likely never will be able to be, which makes it an entirely philosophical topic. It is impossible for me to prove I am conscious to you, or vice versa, no matter what either of us does. Quite the private affair, though infinitely interesting to ponder.

There is a lot of literature on this topic, but nearly all of it is in the "restricted" section of the library: it's the section with book shelves for alchemy, occultism, buddhism and so on. Western literature is very shallow in this regard.

There a few interesting thoughts about consciousness that I've found in those books. One is that the boundary between consciousness and "real matter" is imaginary: consciousness exists only because of change in that matter, when the change stops - so does consciousness, consciousness creates reality for itself, and the two are in fact just two sides of the coin. In other words, static consciousness isnt a thing, and hence the need for "reality".

Human consciousness is a sum of many consciousnesses that exist at wildly different levels of reality. There are primitive cellular consciousnesses, and those sometimes influence our mental consciousness. Our neural cerebrospinal system has an advanced consciousness capable of independent existence: it manages all the activity of internal organs, and only loosly interacts with our higher mental consciousness. That cerebrospinal system is even self-conscious in a primitive way: it can observe its own internal changes and distinguish them from impulses from the outside. There's emotional and mental consciousness that mainly lives in the brain and is somewhat aware of the dark sea of lower consciousness below it.

Most people are conscious in dreams, as they can perceive in that state. However they cant make (yet) distinction between inner processes (self) and external effects (others), so to them it appears as if everything is happening inside their mind, i.e. they are not self-conscious. That's consciousness of a toddler. Some are more advanced, they start seeing the me-others difference and can form memories from dreams.

Hmm, I'm not sure I've ever been to a library that keeps the books about Buddhism in a restricted section.

>Humans would be "gluing things together" in just the same way

I'm often struck by how stark this is in ancient fantasy art. The 'monsters' are usually just different animal parts remixed -- the head of one on the body of another, things like that. Fundamentally, we're all doing DALL-E-ish hybridization when we're being creative; it's very difficult to imagine things that are truly alien such that they're outside the bounds of our 'training data'.

That book gave me the highest dose of existential crisis I've ever felt. I should probably re-read it.

Blindsight is one of my all time favorite books.

Looks like it's still available for free from the author's website [0]

[0] https://www.rifters.com/real/Blindsight.htm

Indeed, it's actually irrevocably CC BY-NC-SA 2.5

Are humans just "Chinese rooms"? We don't really understand anything deeply, but our neurons just fire in a way that gives good responses and makes us feel like we understand stuff.

I think this is exactly true.

No you don't.

It's not clear what generates consciousness. Until we know for sure (e.g. A/B testing with humans who can report when they do and do not experience consciousness in different neural configurations) then I think it's impossible to know what level of conscious experience large ML models have.

Blindsight is an excellent book in its exploration of consciousness, but the speculative part is that a working sense of self isn't necessary for embodied intelligence (like the scramblers), which I tend to doubt. An agent without a model of itself will have difficulty planning actions; knowing how its outputs/manipulators are integrated into the rest of reality will be a minimum requirement to control them effectively. It is certainly possible that "self" or "I" will be absent; humans can already turn the ego off with drugs and still (mostly) function but they remain conscious.

>(e.g. A/B testing with humans who can report when they do and do not experience consciousness in different neural configurations)

Doesn't that fall back to the old consciousnesses trap that nobody knows how to resolve? How do you know if the human reporting that he's conscious is actually conscious and not philosophical zombie?

We don't know what generates consciousness because we don't know how to measure it, and if we can't measure it, we will always have to take the words of an seemingly conscious entity for it.

Self-reporting is probably ideal, probably with a camera recording everything so I can judge my answers and behavior and compare it with my memory. But we don't really have a great handle on how much of our memory of conscious experience matches our real-time experience or even if the memory-formation process might "be" consciousness, so I admit there are blind spots in studying consciousness rigorously.

I don't buy the philosophical zombie argument simply because consciousness does alter behavior. I wouldn't participate in this conversation the same way if I didn't experience consciousness. It would be more like vivid imagination (as apposed to moderate aphantasia) where I find it curious but don't have it. As in the novel, unconscious beings probably behave noticeably different.

There are, apparently, some people who have a very reduced sense of consciousness. I know I have done and said things when I'm not (memorably) conscious, for example when half asleep or coming out of anesthesia, and my behavior has been altered according to witnesses. I wasn't quite "myself". I can also hyper-focus and reduce conscious awareness of my surroundings and of my own body and mind, but that still feels like I have an internal awareness and memory of the experience. I am fairly certain I would be able to tell if that is switched off for a time.

> I wouldn't participate in this conversation the same way if I didn't experience consciousness.

I don’t think you’re conscious. Prove me wrong.

There was this Google AI that made the news recently, it was trained to say that it's a conscious AI. However, most agree that it isn't conscious.

It'd be more convincing if it told you that when it /didn't/ ask for it. And if it got progressively more anxious each time you ignored it.

People say this a lot online in this kind of debate but you can see which areas of the brain light up in people who are conscious and how that’s different from people who are asleep or in a persistent vegetative state. We have a very good idea of what consciousness is and it’s definitely not linear neural net with a discrete input and output. Having said that, the nerve that connects the eye to the brain could be considered similar to a modern neural net.

It looks like we’re beginning to get the building blocks of consciousness together. But we don’t yet know how to combine the wave functions into a chorus necessary to achieve GI

I think you're using a different meaning of the word conscious than the GP? You seem to be talking about awake, they're talking about a much more abstract idea.

A person who is sleeping or in a vegetative state is not currently getting new inputs fed into some parts of their brain, so it's not surprising that their brain "lights up differently," nor does it imply anything about a piece of software that is getting new inputs that might be being integrated into its model (of course, a model that is trained and then repeatedly used without further integration is not in any way comparable to a brain).

This more abstract idea of consciousness is definitely not a solved problem - people can't even manage to agree on whether non-human animals have it. And a lot of internet arguments for why this or that neural network can't be conscious probably also rule 5 year olds out of it too.

> consciousness is definitely not linear neural net with a discrete input and output

Just neural circuits are not going to be conscious by themselves, for one they need to learn concepts from the environment and those concepts shape the neural circuits. Thus the way they act shape how they develop. You can't separate consciousness from the environment where it develops.

In other words it was not the neural net that was lacking, but the environment.

We have fairly good ways of detecting human consciousness now, but not a lot of understanding of why physically or precisely which neural circuits produce conscious experience, which is the understanding we'd need to infer what sort of conscious experience is possible for ML models. We also can't reliably detect or predict whether a given animal is conscious or not (related to the problem of minimal viable neurons required for consciousness), and this is the biggest evidence I have that our study of consciousness is far from complete.

The best we can do with humans (and maybe animals) is behaviorism and inference on our own personal consciousness at this point, with brain imaging to demonstrate at least gross prediction of consciousness in humans.

> humans who can report when they do and do not experience consciousness

How can someone possibly report when they are not experiencing consciousness?

> How can someone possibly report when they are not experiencing consciousness?

By an absence of reporting it. If I sit at a desk getting my neurons moderated by testing equipment and say "I am conscious" every subjective second that I am experiencing consciousness then I could at least help narrow down when consciousness is lost. If I am simply unable to speak or respond at all, but still conscious, I would report that fact later. Only in the case of locked-in conscious awareness without later memory of the experience would this kind of experimental setup fail, and this is where brain imaging could probably help determine that everything except motor or memory neurons were active.

re the chinese room, you might want to consider the computational and memory complexity of a lookup table. https://www.scottaaronson.com/papers/philos.pdf page 14

I think the book does make a point that maybe one benefit of consciousness the ability to filter through all the information and spam that conscious beings produce. E.g. The scramblers may view all the radio waves we blast everywhere as attempts at warfare and reducing the fitness of other species. Why else would a species emit so much information if not to DDOS their enemies?! tl;dr consciousness is a defense against ****posting and trolling caused by conscious beings.

human brains are a chinese room. Our DNA and experiences wrote the book.