I've compared this before to the self-driving car trolley problems that came out a while back. Everyone was so concerned about how the AI would decide to value the human life inside the car vs out of it, who it would hit.

In reality, companies cheaped out on sensors and their cars ran into walls that were painted white and hit minority pedestrians because their training data was woefully inadequate.

Not all giant security risks are sci-fi stories like System Shock. Sometimes they're boring things like "Dave wired up an inherently unpredictable interface to this power plant and just assumed it wouldn't have bugs."

Never mind the fact that solving issues like prompt injection, biased training data, hallucinations, public perception/anthropomorphism, corporate monopolization, worker disenfranchisement, etc... all seem like pretty stinking important steps in building a safe AGI to begin with, even if that is the most important concern long-term.