These same people (I belong to that group), who had a vision back then, to work on a right thing, are now saying clearly that super intelligent machines are nearly there. And likely this is a potential change and a challenge, as we’ve seen many examples when superior intelligence doesn’t care that much about inferior.

As to “become self aware” - it’s super simple to SFT a model that is “self aware”. There is nothing magical in self awareness. There is also not much use in it, so no one is bothering to do it.

Pointing out that it's not inherently AI's fault, more the way we've constructed our society - that the underlying goal is not to sustainably maximize wellbeing but to maximize wealth created (which leads to no safety net to keep the labor pool large and incentivizes the creation of AI to replace people cheaply) - is as important as dispelling "sci-fi self-awareness and autonomy" fears when talking about AI.

And there have been much stupider "public discourse" fads. They're not going anywhere and I'll take what I can get, honestly.

I am more concerned with the human beings using super intelligent algorithms to manipulate people than I am about the algorithms taking on a mind of their own and manipulating us with their own internally generated desires and out-of-control behaviors. But the trope I run in to is people spending all their time talking about the latter, which only serves to shield the former from further scrutiny.

Also I am using fantasy in the narrow sense, meaning something that has not yet occurred which we are imagining. Super intelligence is likely to occur I think, but super intelligence which is beyond our control is much more speculative. Meanwhile powerful people using technology to manipulate people is a story as old as time.