Alternatively, we are analogue beings made up of systems that appear to be best described by chaotic continuous non-linear math, and digital computers can roughly simulate physics to a given level of precision, which can be as fine as you care to make it… at the cost of ever increasing processing requirements. And the amount of physics involved in simulating a single human cell, like a neuron, is already vast.

Whether we can simulate a “large enough” physical system, in as much detail as is required, at a fast enough speed to produce any results in a meaningful timeframe, is an open question.

We don’t even know how big the simulation needs to be, in terms of neurons, to produce qualia. Or how we’d know if it were. Or how much detail/resolution is needed.

It’s like brute forcing the travelling salesman problem for a 10,000 node graph. Yeah, we know exactly how to do it with a digital computer. The math is simple. It’s just going to take longer than the heat death of the universe to get the answer.

Who says computers have to be digital? Analog computers are already a thing.

The preceding discussion was specifically about "a CPU". Analog computers are specifically excluded.

If, despite that, you're trying to simulate an analog entity with an analog computer, or build any analog AGI, the question arises whether it's possible to build one any simpler or faster than existing biological computers, e.g. humans. If no, then why bother?

Considering evolution optimized for biological fitness and not intelligence, it would be incredibly surprising if it wasn't possible to do better, especially using a vastly different architecture.

You could also build an AGI to run on a digital CPU but interact with a simulation that used analog coprocessors (which is a thing these days https://www.youtube.com/watch?v=GVsUOuSjvcg).

You can also include quantum coprocessors (which already exist in their infancy) for various things (probably only useful for quantum simulations at this point though).

Also considering recent ML work, an AGI is more likely going to run on something similar to a GPU than a CPU.

> Considering evolution optimized for biological fitness and not intelligence, it would be incredibly surprising if it wasn't possible to do better, especially using a vastly different architecture.

The way I see it is that humans are the stupidest you can be and still create a society. Like a nuclear reaction, you need average human to add a tiny bit of structure on average and it works, any less than that and society is impossible, any more than that and society forms extremely quickly, so fast that evolution has no time to work.

Given the complexity of the physics apparently involved in brain state, the "and CPUs can simulate physics" needs to do a lot of work for it to be a practical possibility.

Simulating the physics of dopamine on neurotransmitter cells in silicon is an extremely complicated roundabout way of getting a program to run faster in response to certain stimuli, but most alternatives are likely to fall well short of conveying the full sensory experience of human emotion. And its a difficult goal to know you've hit when even simple text-parsing neural networks which don't attempt to model human sensory experience can plausibly mimick descriptions of it...

This cannot possibly be true, because we have no concept of what qualia are.

There is no objective external way to determine if an entity experiences qualia at all. From outside we can only consider behaviour, not subjective experience.

Personally I can't be sure that anyone else experiences qualia. It seems like a reasonable assumption, but I'm basing that on a mapping from my own experiences to those of others.

If everyone else was a highly sophisticated automaton I wouldn't know.

This creates a paradox. The only way to prove qualia exist would be to invent an objective qualia-ometer which would somehow measure subjective experience.

But of course my experience of the qualia-ometer would also be subjective, so the existence of qualia in others cannot be proven objectively.

You can imagine some kind of meta-technology which allowed qualia to be shared, so everyone could experience everyone else's experiences.

Not even that would be absolute proof because experience would still be subjective.

The point being - assuming you can throw cycles at this is looking at it from the wrong end of the telescope. It doesn't work like that because it can't work like that.

It's a much harder problem. (Which could be why it's called the Hard Problem.)

Worse, it just highlights that all of our experiences are qualia. There is nothing objective about anything we experience. What we think of as objective experience is really just stable shared experience.

You can make an argument that the more complicated, stable, surprising, and shared experience is, the more likely it is to be objective.

That's sort-of plausible and possibly a good rule of thumb. But it still doesn't really make the problem go away.

I find it most satisfying to act as if self-consciousness, free will, and qualia are all effectively "illusions" that are emergent properties of sufficiently complex brains, that it's unlikely we'll ever resolve any of the Hard Problems, but that we will be able to construct machines that exhibit as much free will as humans. It doesn't resolve any really important problems, but at least it moves the discussion past "is qualia required for AGI", which clearly it isn't.

Sorry, but I like to work on problems that have solutions and not ones which are so ill-defined that they don't have _any_ satisfying solutions.

Some (some of them in this very comment section) take this and smugly declare qualia don't exist at all.

Your conclusion doesn’t follow from your premises, and your second premise is false besides as others have pointed out — CPUs CANNOT simulate even classical physics exactly, and certainly not quantum physics.

But even if such a complete simulation were possible, there’s every reason to assume a CPU would lack the consciousness to experience anything. When you simulate a hurricane does the CPU get wet?

> your second premise is false besides as others have pointed out — CPUs CANNOT simulate even classical physics exactly, and certainly not quantum physics.

Both classical and quantum physics can be simulated on a classical computer, to an arbitrary degree of precision. Granted, the case in which infinite precision (if such a thing even exists in reality) is required is not simulatable on a discrete computer, but do any experts actually believe this to be the case? It's certainly not an opinion that I've seen around.

I think discussions about "can we actually get enough computing power to do this in practice" are beside the point - the discussion was about whether computers can feel in principle. If we wanted to do it in practice and were at the point where this was feasible, we'd probably engineer a CPU or co-processor more suited to the task than the general-purpose CPUs of today.

> there’s every reason to assume a CPU would lack the consciousness to experience anything.

If we are physical beings, then "consciousness" and anything else we have must be an emergent property of our physical components. If we can simulate those physical components, then this simulation will exhibit the same properties - consciousness and anything else one can attribute to us.

If our consciousness comes from non-physical properties we have (a "soul" or anything metaphysical), then sure, I'd agree with you.

> If we are physical beings, then "consciousness" and anything else we have must be an emergent property of our physical components. If we can simulate those physical components, then this simulation will exhibit the same properties - consciousness and anything else one can attribute to us.

Again, a simulation is not the thing. The map is not the territory. If consciousness truly emerges from actual physical processes of interacting brain matter (seems plausible), those _don’t exist_ in a computer simulation.

In a simulation of a brain, from what substrate could consciousness emerge? The state of the simulated brain is stored in an arbitrary subset of locations in RAM, unknown to and non-interactive with each other, along with loads of other stuff the computer is keeping track of. Do you think consciousness could emerge automatically from the state of the right subset of locations in RAM, or is it whenever a relevant value in memory is changed due to a transistor opening, or is it when the simulation computation that will result in the RAM update is happening, or is complete? Per the Chinese Room argument, would consciousness still emerge if half the operations were actually performed off-CPU by human mechanical turkers with rule books and notecards? Nothing in the abstract computation will have changed.

Consider also that physical reality runs in full parallel, while simulations on computers run serially per core. So if consciousness emerging requires the simultaneous interaction of many moving brain parts, that isn’t something that happens in a computer simulation.

> Both classical and quantum physics can be simulated on a classical computer, to an arbitrary degree of precision

Quantum physics can’t be simulated on a classical computer to an arbitrary degree of precision. Feynman didn’t think so, and he hasn’t been gainsayed yet. And classical physics is full of chaos and very sensitive to precision.

> In a simulation of a brain, from what substrate could consciousness emerge?

Exactly the same substrate as our brains are derived from: physical particles and their interactions, perfectly replicated inside the simulation. If the simulation is accurate enough, the real particles and the simulated particles behave exactly the same, hence they produce the same results.

> Do you think consciousness could emerge automatically from the state of the right subset of locations in RAM

Hard question to answer since consciousness is hard to analyse. But we can turn it around into a question whose answer is the same, with a bit of rephrasing:

Do you think consciousness could emerge automatically from the state of the right subset of particles in our physical world, or is it whenever a relevant particle state are changed due to particles interacting according to the laws of physics, etc etc

> Consider also that physical reality runs in full parallel,

We don't really know this to be the case. It looks like that to us, but that could easily be an illusion created by mechanisms we can't observe. Just as characters in a video game can't observe how their world is simulated - everything is perfectly consistent whether it was calculated in one CPU thread or several.

Or a more succinct question: why do you think a simulation of consciousness is the same as consciousness? What other simulations of things are identical with the things?

I think that a sufficiently accurate simulation of a system exhibits the same emerging properties as the system itself.

For example, if I can perfectly simulate the weather in some simple planet, all possible emerging weather phenomena for that planet (say like clouds, rain etc) will be perfectly replicated in the simulation. Similarly, if we can perfectly simulate a human body, all of the emerging human phenomena will exist in the simulation (muscle movement, nerve impulses, brain patterns resulting in consciousness etc). I don't think consciousness is fundamentally different from other physical phenomena, it's just a particularly complex example.

Another angle to think about: We can't prove that we're not living in a simulation (or can you?). So our consciousness itself might be simulated for all we know. This is not a proof that we are amenable to being simulated, but it means that disproving it is very hard or impossible.

You lost me. You think consciousness is a _physical_ phenomenon that would necessarily emerge from an accurate _simulation_ of a particle system? If it’s a physical phenomenon in reality, then just like the clouds and rain in your weather sim aren’t physical, only a simulation of consciousness will be present in your simulation.

Feel free to replace "physical phenomena" with "phenomena caused by physics laws" if it makes more sense that way.

What is every reason to assume consciousness has an astral component?

We can definitely simulate things approximately, but exact simulations are another matter entirely, and how much do you lose in the approximation? PBS spacetime discussed the intractability of simulating even one atom exactly: https://www.youtube.com/watch?v=55c9wkNmfn0

It's probably a good idea to have at least a rough physical definition of "qualia" and empirically validate it, without that one can't judge if a simulation is accurate or not.

If an AGI has a body in a simulation, it has a body. If such a thing is a necessary condition for general intelligence then the author's hypothesis is correct.