Re:The Simulation Argument and the Simulation Barrier
by antonvs on Sunday November 24, 2019 @06:57PM
(#21624413)
There's a problem with that description which the author implicitly acknowledges with his use of language:
> "...the agent could be called self-aware."
"Could be called" being the key phrase. Yes, you can call such a model self-aware, because indeed it incorporates a model of itself so it is self-aware in a very literal sense.
However, there's no reason to believe that a model with the ability to represent itself would have conscious experience of the kind humans do. Why should we expect it to be any different from any other ML model or computer program, just mechanically executing its program without any conscious experience? What stops it from being, basically, a machine version of a philosophical zombie?
This is the hard problem of consciousness in a nutshell, and the Schmidhuber comment doesn't address it in any meaningful way.
> "...the agent could be called self-aware."
"Could be called" being the key phrase. Yes, you can call such a model self-aware, because indeed it incorporates a model of itself so it is self-aware in a very literal sense.
However, there's no reason to believe that a model with the ability to represent itself would have conscious experience of the kind humans do. Why should we expect it to be any different from any other ML model or computer program, just mechanically executing its program without any conscious experience? What stops it from being, basically, a machine version of a philosophical zombie?
This is the hard problem of consciousness in a nutshell, and the Schmidhuber comment doesn't address it in any meaningful way.