Do you think we're metaphysical beings, or that physics are beyond the capacity of a CPU to simulate?

Whether we can simulate a “large enough” physical system, in as much detail as is required, at a fast enough speed to produce any results in a meaningful timeframe, is an open question.

We don’t even know how big the simulation needs to be, in terms of neurons, to produce qualia. Or how we’d know if it were. Or how much detail/resolution is needed.

It’s like brute forcing the travelling salesman problem for a 10,000 node graph. Yeah, we know exactly how to do it with a digital computer. The math is simple. It’s just going to take longer than the heat death of the universe to get the answer.

If, despite that, you're trying to simulate an analog entity with an analog computer, or build any analog AGI, the question arises whether it's possible to build one any simpler or faster than existing biological computers, e.g. humans. If no, then why bother?

You could also build an AGI to run on a digital CPU but interact with a simulation that used analog coprocessors (which is a thing these days https://www.youtube.com/watch?v=GVsUOuSjvcg).

You can also include quantum coprocessors (which already exist in their infancy) for various things (probably only useful for quantum simulations at this point though).

Also considering recent ML work, an AGI is more likely going to run on something similar to a GPU than a CPU.

The way I see it is that humans are the stupidest you can be and still create a society. Like a nuclear reaction, you need average human to add a tiny bit of structure on average and it works, any less than that and society is impossible, any more than that and society forms extremely quickly, so fast that evolution has no time to work.

Simulating the physics of dopamine on neurotransmitter cells in silicon is an extremely complicated roundabout way of getting a program to run faster in response to certain stimuli, but most alternatives are likely to fall well short of conveying the full sensory experience of human emotion. And its a difficult goal to know you've hit when even simple text-parsing neural networks which don't attempt to model human sensory experience can plausibly mimick descriptions of it...

There is no objective external way to determine if an entity experiences qualia at all. From outside we can only consider behaviour, not subjective experience.

Personally I can't be sure that anyone else experiences qualia. It seems like a reasonable assumption, but I'm basing that on a mapping from my own experiences to those of others.

If everyone else was a highly sophisticated automaton I wouldn't know.

This creates a paradox. The only way to prove qualia exist would be to invent an objective qualia-ometer which would somehow measure subjective experience.

But of course my experience of the qualia-ometer would also be subjective, so the existence of qualia in others cannot be proven objectively.

You can imagine some kind of meta-technology which allowed qualia to be shared, so everyone could experience everyone else's experiences.

Not even that would be absolute proof because experience would still be subjective.

The point being - assuming you can throw cycles at this is looking at it from the wrong end of the telescope. It doesn't work like that because it can't work like that.

It's a much harder problem. (Which could be why it's called the Hard Problem.)

Worse, it just highlights that all of our experiences are qualia. There is nothing objective about anything we experience. What we think of as objective experience is really just stable shared experience.

You can make an argument that the more complicated, stable, surprising, and shared experience is, the more likely it is to be objective.

That's sort-of plausible and possibly a good rule of thumb. But it still doesn't really make the problem go away.

But even if such a complete simulation were possible, there’s every reason to assume a CPU would lack the consciousness to experience anything. When you simulate a hurricane does the CPU get wet?

Both classical and quantum physics can be simulated on a classical computer, to an arbitrary degree of precision. Granted, the case in which infinite precision (if such a thing even exists in reality) is required is not simulatable on a discrete computer, but do any experts actually believe this to be the case? It's certainly not an opinion that I've seen around.

I think discussions about "can we actually get enough computing power to do this in practice" are beside the point - the discussion was about whether computers can feel in principle. If we wanted to do it in practice and were at the point where this was feasible, we'd probably engineer a CPU or co-processor more suited to the task than the general-purpose CPUs of today.

> there’s every reason to assume a CPU would lack the consciousness to experience anything.

If we are physical beings, then "consciousness" and anything else we have must be an emergent property of our physical components. If we can simulate those physical components, then this simulation will exhibit the same properties - consciousness and anything else one can attribute to us.

If our consciousness comes from non-physical properties we have (a "soul" or anything metaphysical), then sure, I'd agree with you.

Again, a simulation is not the thing. The map is not the territory. If consciousness truly emerges from actual physical processes of interacting brain matter (seems plausible), those _don’t exist_ in a computer simulation.

In a simulation of a brain, from what substrate could consciousness emerge? The state of the simulated brain is stored in an arbitrary subset of locations in RAM, unknown to and non-interactive with each other, along with loads of other stuff the computer is keeping track of. Do you think consciousness could emerge automatically from the state of the right subset of locations in RAM, or is it whenever a relevant value in memory is changed due to a transistor opening, or is it when the simulation computation that will result in the RAM update is happening, or is complete? Per the Chinese Room argument, would consciousness still emerge if half the operations were actually performed off-CPU by human mechanical turkers with rule books and notecards? Nothing in the abstract computation will have changed.

Consider also that physical reality runs in full parallel, while simulations on computers run serially per core. So if consciousness emerging requires the simultaneous interaction of many moving brain parts, that isn’t something that happens in a computer simulation.

> Both classical and quantum physics can be simulated on a classical computer, to an arbitrary degree of precision

Quantum physics can’t be simulated on a classical computer to an arbitrary degree of precision. Feynman didn’t think so, and he hasn’t been gainsayed yet. And classical physics is full of chaos and very sensitive to precision.

Exactly the same substrate as our brains are derived from: physical particles and their interactions, perfectly replicated inside the simulation. If the simulation is accurate enough, the real particles and the simulated particles behave exactly the same, hence they produce the same results.

> Do you think consciousness could emerge automatically from the state of the right subset of locations in RAM

Hard question to answer since consciousness is hard to analyse. But we can turn it around into a question whose answer is the same, with a bit of rephrasing:

Do you think consciousness could emerge automatically from the state of the right subset of particles in our physical world, or is it whenever a relevant particle state are changed due to particles interacting according to the laws of physics, etc etc

> Consider also that physical reality runs in full parallel,

We don't really know this to be the case. It looks like that to us, but that could easily be an illusion created by mechanisms we can't observe. Just as characters in a video game can't observe how their world is simulated - everything is perfectly consistent whether it was calculated in one CPU thread or several.

For example, if I can perfectly simulate the weather in some simple planet, all possible emerging weather phenomena for that planet (say like clouds, rain etc) will be perfectly replicated in the simulation. Similarly, if we can perfectly simulate a human body, all of the emerging human phenomena will exist in the simulation (muscle movement, nerve impulses, brain patterns resulting in consciousness etc). I don't think consciousness is fundamentally different from other physical phenomena, it's just a particularly complex example.

Another angle to think about: We can't prove that we're not living in a simulation (or can you?). So our consciousness itself might be simulated for all we know. This is not a proof that we are amenable to being simulated, but it means that disproving it is very hard or impossible.

It doesn't need fingernails but it absolutely needs the ability to effect change in the world it is meant to be "intelligent about" and learn from the results.

Posting this is a bit of trying to hold back the tide, because the dominant threads in HN AI/BI discussions are almost always horribly uninformed, but I'm posting this so that if anyone else who actually does work in those fields is reading this and wondering what on earth is going on, now you know you're not alone!

Now, if we want to model the world (or build "world model", in LeCun's words), I am not sure "interacting" with the world is a necessary condition given the amount of data we already have. GPT-3 probably has some vague "world model" inside these parameters and we don't know how to effectively extract it out.

The mind has a physical structure that supports it. We are still unable to understand a meaningful portion of that structure or replicate it.

Creating a concept to speculate about a system that we have no objective knowledge, based on subjective experience is useless. The name is catchy for a song though.

Why would moving physical objects like electrons by micro meters not be good enough, but by meters it would?

But the question "Can a CPU experience qualia?" can not be answered by saying:

"no, because CPU computation exists on some platonic ideas plane".

Computation is physical both in brain and in silica.

Does that mean that embodiment is required? No. But your definition of "embodied" misses the point of the discussion.

But that isn't what Isinlor was talking about. They said "moving electrons around" within the normal CPU circuitry as if that was the same as actuators controlling the external world. It's not.

In fact a single transistor is both a sensor and actuator.

In fact you can use them to do things they were not designed to do like trough row hammer:

https://en.wikipedia.org/wiki/Row_hammer

CPUs can also detect things like collisions with cosmic rays.

CPUs just can't produce macroscopic forces relevant on human scale.

The baseline assumption in modern neurology is that meaning, for living things, is derived from their physical experience of the world. In short that no paper definition can ever substitute for experiencing first-hand what the "effect" of something is, what it does to you or to the world and what that entails for your body, your life. That is what "knowing" and "meaning" ultimately are.

These experiences are the basis for associations in our constant cognitive (and emotional, instinctive, sensory, symbolic, etc) modelling of the world. Associations beget complexity (dimensionality, modalities) of the model. Hence, "finely tuned" or "high definition" i.e. ever more "advanced" intelligences.

Given the overall crudeness and extreme simplicity of a CPU (or any man-made machine) compared to any living organism, I'm actually willing to bet that should CPU-based AI's "feel" something, it's on the order of 1/100th of what a shrimp feels, and they'd develop real-world intelligence at that level. Infinitely smarter than a potato, yet infinitely dumber than a fish. You don't run ape software on amoeba hardware.

Transistor is a sensor. It detects voltage on the base pin. Transistor density and so sensor density on a modern CPU is really high. In fact modern CPU transistor density is 3 orders of magnitude higher than nerve density even on your retina.

It's at the order of ~100,000 rods/mm^2 for retina and ~100,000,000 transistors/mm^2 for modern CPU.

The ghosts in Super Mario World are "intelligent" and "embodied" in Super Mario World, but they are not "embodied" on Earth and thus cannot be "intelligent" there.

The "G" in AGI implies intelligence on Earth, not in a video game.

Asking whether a CPU can experience qualia is kind of like asking if atoms can experience anything. The CPU is just the substrate on which software runs.

For that matter, does a brain have a subjective experience, or is it just the substrate on which a mind runs?

I won't say I'm at that point with the question of qualia, but I share the other commenter's reflexive eye-rolling when I hear it invoked. Perhaps the question doesn't deserve to be dismissed, but I'd certainly like to see some new clarity about what the question is really asking, what problems would be solved by finding an answer to the question, or heck, even a remotely plausible description of what form a valid answer might take.

The question is simply "Will robots feel just like you do, or will they merely be pretending?" - it's something any child could understand. Answers could be something like "No, because they lack X", "Yes, but only those that have X" or "Yes, everything does (panpsychism)".

Do you care about potential AI suffering? If you care about animal suffering, the answer to the question would have pretty obvious and major ethical implications.

Now, I'm not convinced that humans can ever determine an answer, so I can understand thinking it's just navel gazing, but I imagine the same applies to knowing what happened before the big bang. I think it's quite evident that this doesn't mean either question is malformed.

But what is the epistemological status of these questions? That’s what I’m concerned with, not whether children can understand and be fascinated with these questions.

Unless you're religious, the epistemic status of "do dogs go to heaven?" is obviously "no" (or "we have no way of knowing"). If you think the answer to "can robots experience qualia?" is equally trivial, then just say so, instead of going on some spiel about how actually the question is unclear and nonsensical when it pretty obviously isn't.

If I can't be 100% sure that people feel in the same way that I do, then asking that same question for robots is totally pointless.

>If I can't be 100% sure that people feel in the same way that I do, then asking that same question for robots is totally pointless.

I don't see how that at all follows.

IMHO it's the old darkness/duality problem all over again — postulating some "essence" (e.g. the spiritual/religious Soul) and basing theory on a mere intuition, instead of invoking a "placeholder" (like "dark matter" isn't necessarily matter but literally "something that behaves as such" whatever it actually is, or like "dark energy" isn't necessarily a fifth interaction but certainly behaves as such or as anti-gravity of sorts; wherein "dark" means "unknown and hidden from direct observation, merely inferred"). Hence my call to not infer the very measurement thereof if we are to call an explanatory hypothesis.

Should we start by stating that "qualia" or "consciousness" or whatever is a placeholder name for "whatever dark (unknown, black-box) mechanism explains the observable phenomenon named as such", then we might have a chance at actually forming a coherent theory thereof. IMHO.

Perhaps, although in the case of qualia I’m not sure what the phenomenon is that we need a placeholder explanation for. For dark matter we’ve got certain observations of galaxies. For qualia we’ve got, what? A lot of people who feel very strongly that they are unable to deny the existence of something that they cannot describe beyond that claim of undeniable existence? If that’s how astrophysicists explained dark matter I’d be inclined to think that maybe it just doesn’t exist.

Edit: thanks to autocomplete we were briefly embroiled in an argument about the existence of quails.

Seems to me that the bafflement I'm experiencing at people doubting human sensory experience is the real bit, and the neural network that might be able to perfectly replicate my bafflement in silicon is the purely hypothetical construct.

In theory, if we had high-throughput brain-brain interfaces that could be connected between your thalamus and someone else's, then yes, they could experience the same sensations as you. Of course, besides the challenge of having high-throughput hardware embedded in your brain without killing you, there's also the challenge that brains don't have exact correspondences between every neuron for different people, so any such mapping would be quite complex.

But there is an actual example of a pair of conjoined twins who share the same thalamus and can experience the same sensations as one another: https://en.m.wikipedia.org/wiki/Krista_and_Tatiana_Hogan

> it was confirmed that they share a thalamus which connects their brainstems. Through this shared brain tissue structure and the interconnected neurons, one brain receives signals from the other brain and vice versa. This documentary also reported on experiments that were carried out that confirmed that visual cortex signals based on what one girl saw, were received by both girls' brains. So in effect, one twin could see what the other twin was seeing, making them unique even among craniopagus twins.

Also...

> I'm also certain my sensory experiences are not identical to attempts to quantify them or describe them in language or interpretive dance (philosophers struggling to define and articulate things even at high levels of abstraction is weak evidence in favour of qualia, since the defining aspect of qualia is its the portion of human brain state that can't be communicated).

Language is a very low-fidelity way to express information; we can use abstractions to make up for quite a bit, but it doesn't make sense to think that we're accurately transmitting what's going on in every molecule of our bodies when we say what something feels like to us. And even if we could send an accurate account of all those details, who would have the time to parse through and make sense of it all? And that would be a momentary expression of who we are; it'd be inaccurate in the next moment.

That's sort of the point though, it's incredibly low fidelity (so low fidelity that computers we're absolutely sure don't have sensory experiences can appropriately and plausibly describe them) yet just about high fidelity enough for me to be sure that people experiencing synesthesia must be having different experiences than me, and there's no particular reason to believe we are or ever will be on a path to me being able to experience their experience in any meaningful way. Even in the twins' case where the transmission mechanism is literally the same substrate connected to the same physical organs that have developed simultaneously, the reports of them having different personalities and preferences suggests that they have different sensory experiences from [partially] shared inputs.

The thorny question then becomes whether sensory experiences are important to independent goal formation and other stuff often considered part of general intelligence. I don't think we know enough to answer that question, but we certainly don't know enough to dismiss it.

Related: https://en.wikipedia.org/wiki/Philosophical_zombie

The answer is that one of these things is true: * P-zombies would write IDENTICAL papers about how they experience qualia but p-zombie land wouldn't * The mind has some effect on the universe you can detect, so P-zombie universe looks quite different

So what? Software is still physical configuration on a running system.