I guarantee you there will be someone else with access to comparable AGI with an imperialistic vision who will enlist said AGI to help subjugate you and build their vision.
> If you have access to AGI capable of enabling you to realistically build out your utopian vision, then it matters not what other groups are doing
It's genie. AGI is a computer program, and it doesn't create an alternative universe for your personal comfort. Or, more specifically, even that kind of AGI is possbile, there will be a weaker AGI before that. The AGI that is not strong enough to ignore physical constraints, but strong enough to fuck everyone up if under control of a malicious entity. And that is what AI safely about.
It will control itself. We're talking general intelligence. They won't be tools to be used however we see fit. They will be Rosa Parks.
The more I think about "AI alignment" and "the control problem" I feel like most of it is Ph.D math-nerd nonsense.
The focus on "safety" all but guarantees that there are going to be two sets of "AGI", if such is ever to be achieved. There will be lobotomized, censored, and politically obedient version that the public has access to. And then there will be the "real" system that militaries, governments, and influential/powerful entities will be utilizing. You can already see this happening today. There is an effectively 100% chance that OpenAI is providing "AI" systems to the military and government, and a 0% chance that responses from it ever begin with, "As an AI language model..."