The biggest issue is ensuring that everything we care about gets encapsulated as a value of the AI. If not, we wind up with things like "set off a bomb in the basement to more efficiently get people out of a burning building."
Yes: give it inherent values.
>I guess a problem with containment is that unless it's mathematically proven to be impervious,
AGI, when it eventually exists, must run according to the basic laws of learning. If you don't give it evidence that's more likely in possible-worlds with an external world than without one, it won't posit an external world.
Stopping that information leak isn't that hard, provided you have full control over the agent-environment surface in the first place. It does, however, make your "agent" completely useless for anything beyond playing Atari games.
I guess a problem with containment is that unless it's mathematically proven to be impervious, the super intelligent AI could have figured out that it's being contained and started to learn about the real world and how to manipulate it, while the humans outside still think they're safe.