There's a problem with that description which the author implicitly acknowledges with his use of language:

> "...the agent could be called self-aware."

"Could be called" being the key phrase. Yes, you can call such a model self-aware, because indeed it incorporates a model of itself so it is self-aware in a very literal sense.

However, there's no reason to believe that a model with the ability to represent itself would have conscious experience of the kind humans do. Why should we expect it to be any different from any other ML model or computer program, just mechanically executing its program without any conscious experience? What stops it from being, basically, a machine version of a philosophical zombie?

This is the hard problem of consciousness in a nutshell, and the Schmidhuber comment doesn't address it in any meaningful way.

I thought i read something from him which was more esoteric when he was describing a soul but can't find anything.

Your link is quite technical.

Thanks for this, it's new to me and that entire ama is fascinating.

edit: Out of curiosity, why would one consider him infamous?

He's known by many in the AI/ML community as someone who likes to slay down claims of novelty and likes to dig up connections to decades old ideas (which heavily include his own works). He's controversial, meaning that many people like him and many dislike him, few are neutral.

Perhaps look up the case with Ian Goodfellow's work on GANs. Schmidhuber said the GAN paper doesn't give sufficient credit to his earlier work on predictability minimization and was very vocal about this also in conferences (interjecting etc.).

>Out of curiosity, why would one consider him infamous?

IIRC claims were made that LeCun etc. were unfairly credited for work pioneered by him. As in, he pretty much invented modern ML but wasn't credited at all.