I don't think there's a way to avoid that, really, because AI will be better than us at everything, so we will want to give everything over because it will make sense, and nation-states who don't will fall behind.
The question will be if we have safety mechanisms if AI starts going rogue.
I agree. There is a very large moat between where we are now and the envisioned AI that people hope for. And that moat is filled with lots of problems without any clear answers.
> because AI will be better than us at everything
Yes, I don't think this concept still has not been completely digested by most people. AI is essentially a skill/technology replication machine. It is going to bring about a crisis of meaning in society as well.
> The question will be if we have safety mechanisms if AI starts going rogue.
I've spent a lot of time thinking about and writing on this topic as well. Under the current premise of alignment theory, I believe it is a complete logical fallacy or paradox. I don't see it as a hard problem, but rather an impossible problem.
Now, this is just from the perspective of taking alignment theory's own proposed conditions. That being AI will be power seeking, scale infinitely on its own and we will solve this by applying humanities values.
I explain why I perceive this as an impossible solution here.
https://dakara.substack.com/p/ai-singularity-the-hubris-trap
Second, I wonder if we could force conversation-points where AI talks to other AI's to be simple and perhaps open-source.
So a AI banking API talking to the Tesla API can never say much more than "yes this driver has funds" etc. It can't talk about world domination.
And even if we were hypothetically successful, and has been demonstrated already with the release of primitive LLMs, someone out there will build unaligned AI purposely.
I was just thinking about North Korea, and how AI will enable statewide surveillance on a whole new level, but then the thought popped into my head that maybe if AI can't be stopped, it can't be stopped there either, and maybe it'll be benevolent, and in 30 years they will be freed. ¯\_(ツ)_/¯ Hard to know how this plays out.
Hoping it will be benevolent is about the only thing we have at this point. Many, but not all, alignment safety researches believe that it will inherently exhibit properties that make for disastrous outcomes. Since they have no solution for this problem if it exists, at this point hoping they are wrong is the best option.
For the life of me I can't see how, or why, we would give AI this sort of power. Look at the power of the algorithm in social networks and see how that has taken shape. This embrace of the machine just seems so short-sighted.
Yes, great point, I also mention that very aspect in the article linked previously.
AI will likely surpass social media by orders of magnitude in both its ability to provide positive value for individuals as well as its propensity to become the dark monolith of a dystopian society under the control and manipulation of forces unseen
I think there is a perception that the machine will be able to "rise above" our own biases and limitations. However, I see that also as another fallacy for which I elaborated further here.
The Paris Morgue doesn’t still have corpses on display for the public.
“Death as Entertainment at the Paris Morgue”
https://www.atlasobscura.com/articles/paris-morgue-public-vi...
Some suggest we can use AI to counter these itself, although I haven't seen much clarity on such proposals and even if possible there is always substantial lag between nefarious use and containment.
There is so much focus on the problems of future AGI, but little on AI that we have now, working as designed, but yet still very problematic from the impacts on societal order.
I've elaborated much on the societal and social implications in the following reference. I expect AI will lead all concerns in the area of unexpected consequences in due time.
https://dakara.substack.com/p/ai-and-the-end-to-all-things