Do you know what really smart people do? They don't do any of the things that AI Risk says will happen, they get jobs at Google (or insert other pointless white collar job on wall street) moving widgets by 1 pixel to drive an ad engagement metric by half of a percentage and get paid $500k+ a year in total compensation. Why? Because it is a relatively low effort way to fund a lifestyle that allows enjoyment of your intellect through experiencing the world in other dimensions besides the mundane of work.
The non-biological aspect is the only reason why AI Risk has any credence at all /in our current state/, because AI isn't distracted by a desire to travel the world and explore nature, or to read all the great works of literature and write their own analysis of it, etc. Biological beings, at least at this point in time and in the probable sense, have emotions and therefore have desires that range beyond whatever machinations you expect from AI Risk.
But something else to consider is that so far biological super intelligence doesn't seem to get all that much smarter than what humans are at generally. If somebody is guarded against you, then it's very difficult (impossible) to fool them even if you are way smarter than they are. This could indicate that maybe there's a cap to how super intelligent AI can get as well.
It’s the same thing lately with AI… Sure, it has slightly more merit than cryptocurrency, but it doesn’t do what people think it actually does — certainly not as well either.
AI is not sentient yet people write full paragraphs to it as if it were? They even expect such in response. This is LUNACY! This is a mass delusion! Real life is not a Star Trek TNG episode with a talking computer!
It’s not an accident either. Nobody wants to look like a fool when there is a lot of money at stake. So the fly by night snake oil salespeople thrive… Some technology will be developed by overly sincere people along the way, but that was never the real goal.