I am not sure I understand exactly what you are trying to argue so I will re-state my concerns.

I am more concerned with the human beings using super intelligent algorithms to manipulate people than I am about the algorithms taking on a mind of their own and manipulating us with their own internally generated desires and out-of-control behaviors. But the trope I run in to is people spending all their time talking about the latter, which only serves to shield the former from further scrutiny.

Also I am using fantasy in the narrow sense, meaning something that has not yet occurred which we are imagining. Super intelligence is likely to occur I think, but super intelligence which is beyond our control is much more speculative. Meanwhile powerful people using technology to manipulate people is a story as old as time.